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A  TENTATIVE  STANDARDIZATION  OF  A 
HARD  OPPOSITES  TEST 


The  Problem  and  its  Importance 

There  have  been  many  attempts  to  apply  mental  tests  to  college 
students,  but  the  results  have  not  been  on  the  whole  satisfactory. 
The  very  homogeneity  of  the  college  group,  brought  about  by  the 
natural  process  of  the  elimination  of  the  unfit,  should  have 
warned  us  not  to  expect  results  comparable  to  those  obtained 
from  children  and  unselected  adults.  The  minute  individual 
differences  of  degrees  of  accuracy  or  rate  of  performance  can  be 
detected  only  by  the  use  of  a  most  delicate  scale. 

Therefore  to  raise  our  correlations  of  mental  tests  and  esti- 
mated intelligence  it  is  necessary  that  greater  attention  should  be 
paid  to  the  selection  of  the  tests  themselves.  Where  the  higher 
functions  are  involved  it  is  better  to  avoid  tests  of  co-ordination 
and  sense  discrimination  as  well  as  those  tests,  success  in  which 
is  largely  dependent  upon  speed  of  performance  rather  than  ac- 
curacy. Unfortunately  the  humbler  task  of  standardization  for 
convenient  use  has  been  neglected  and  until  investigators  have 
at  their  disposal  a  number  of  such  tests,  our  attitude  toward  the 
validity  of  mental  testing  of  college  students  should  be  highly 
tentative. 

This  present  investigation  was  attempted  in  the  hope  of  con- 
tributing a  small  amount  toward  the  preparation  of  some  ade- 
quately standardized  tests.  To  evolve  out  of  a  test  in  common 
use  a  vastly  more  difficult  test,  to  determine  the  relative  dif- 
ficulty of  its  parts,  to  standardize  the  responses  which  are  accept- 
able, and  to  set  up  workable  norms  with  which  comparisons 
could  be  made — this  specifically  was  our  aim.  For  this  purpose 
we  have  chosen  an  opposites  test.  No  efifort  has  been  made  to 
obtain  correlations  with  other  tests  and  no  claim  is  made  that 
the  test,  as  it  now  stands,  has  mentally  diagnostic  values  of  high 
importance  for  the  individual. 


II. 

The  Results  Obtained  by  Previous  Investigators 

The  diversified  use  of  the  opposites  test,  with  its  different 
methods  of  presenting  and  scoring,  has  yielded  results  which, 
while  not  altogether  comparable,  are  not  without  interest. 

Dr.  Bonser^  in  an  investigation  of  the  reasoning  ability  of 
children  in  the  upper  division  of  the  fourth,  fifth,  and  sixth 
grades  of  the  public  schools  of  Passaic,  New  Jersey,  tested  three 
hundred  and  eighty-five  boys  and  three  hundred  and  seventy-two 
girls  within  the  period  of  February  13  to  27,  1906.  The  tests 
employed  were  designed  to  exercise  the  most  fundamental  phases 
of  reasoning  ability,  namely,  mathematical  judgment,  controlled 
association,  selective  judgment,  and  the  analytic  and  synthetic 
thinking  necessary  for  the  intellectual  interpretation  of  litera- 
ture. 

For  controlled  association,  three  types  of  tests  were  used. 
First,  in  two  sets  of  ten  sentences  presented,  a  significant  word 
was  omitted  from  each  sentence,  which  was  to  be  filled  in  by 
the  pupils.  Second,  two  other  sets  of  ten  sentences  were  given,  in 
each  of  which  two  significant  words  were  placed  one  above  the 
other.  The  pupils  were  instructed  to  draw  a  line  through  the 
wrong  word,  leaving  the  sentence  so  it  would  read  correctly. 
Third,  three  sets  of  twenty  words  each  were  given  to  the  pupils 
with  the  instructions  that  they  were  to  write  beside  each  re- 
spective word  a  word  just  its  opposite  in  meaning. 

For  selective  judgment  two  types  of  tests  were  used.  The 
first  consisted  of  two  sets  of  two  series  each  of  ten  reasons  why 
some  given  fact  is  true,  some  of  which  are  correct,  the  others 
incorrect.  The  pupils  were  to  check  the  former.  Second,  there 
were  given  two  sets  of  three  series  each,  of  five  definitions  of  a 
given  thing  or  term,  some  of  which  were  correct,  others  incor- 

1  Bonser,  Frederick  G.,  "The  Reasoning  Ability  of  Children  of  the  Fourth, 
Fifth,  and  Sixth  School  Grades,"  Col.  Univ.  Conl.  Ed.,  t,~,  1906,  i-ioi. 
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rect  or  irrelevant.  The  pupils  were  to  choose  the  right  definition. 
In  the  table  below  these  tests  are  indicated  as  V  and  VI  respect- 
ively. 

For  literary  interpretation  the  pupils  were  asked  to  write  the 
meaning  of  two  stanzas  of  poetry. 

Below  is  shown  the  coefficients  of  correlation  obtained  by  cor- 
relating each  test  with  the  combined  score  of  all  the  tests. 

TABLE  I 

Bonser's  Correlations  of  Each  Test  with  the  Totals  for  All  Tests 

in  Reasoning  Ability. 

Opposites  and  Combined  Score   

Selective  Judgment  and  Combined  Score   (V) 

Mathematical  Judgment  and  Combined  Score 

Selective  Judgment  and  Combined  Score  (VI) 

Controlled  Association  and  Combined  Score 

Interpreting  Poems  and  Combined  Score   

Spelling  and  Combined  Score  


85 
73 
59 
58 
55 
37 
22 


Dr.  Bonser  has  arranged  for  comparison  separate  tables  for 
the  younger  and  the  older  group  in  each  grade.  He  finds  the 
younger  group  superior  in  the  opposites  test  for  every  grade,  but 
this  superiority  diminishes  as  the  years  in  school  increase.  This 
would  indicate,  he  thinks,  that  the  test  is  one  which  reveals  some 
sort  of  native  ability,  which  is  concealed  in  the  upper  grades 
where  the  test  is  relatively  simple. 

Miss  Norsworthy"  in  a  study  of  the  comparison  of  defective 
and  normal  children  found  that  the  defectives  were  farthest 
removed  from  normal  children  in  ability  to  deal  with  abstract 
data.  Of  the  137  cases  ranging  in  age  from  eight  years  up,  none 
of  the  defectives  surpassed  the  median  score  for  normal  children 
in  the  opposites  tests.    This  is  shown  in  the  following  table. 

Woolley  and  Fischer^  in  their  work  in  connection  with  the  in- 
dustrially employed  children  in  Cincinnati  obtained  results  by  the 
use  of  opposites  tests  which  are  highly  significant.  Over  eight 
hundred  children  fourteen  years  of  age  were  given  the  following 

2  Norsworthy,   Naomi,  "The   Psychology  of   Mentally   Deficient   Children," 
New  York,  Columbia  University,  1906. 

3  Woolley,   Helen   Thompson,   and    Fischer,   Charlotte    Rust,    "Mental    and 
Physical    Measurements    of    Working    Children,"    Psychol.    Monog.,    1914, 

XVIII,  77,  213-241- 
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TABLE  II 

Norsworthy's  Percentages  of  Children  Scoring  above  —2  P.  E.,  — i  P.  E., 

and  Median  for  Normal  Children  in  Series  of  Tests. 

%  above — i  %  above  — 2 

%  above  Med.      P.  E.  (or  P.  E.  (or 

for  ordinary        lowest  25%  lowest  9% 

children            of  ordinary  of  ordinary 

children)  children) 

Height    45  61  77 

Weight  44  66  77 

Pulse    49  69  86 

Temperature    26  59  77 

Weight  Test 18  28  39 

A-T  Test I  14  28 

Memory  of  Unrelated  Words  6  18  27 

Dictation    10  10  21 

Memory  of  Unrelated  Words  5  19  30 

Part-Whole  Test   9  17  27 

Genus-Species  Test  9  16  17 

First  Opposite  Test .0                          0.9  5 

Second  Opposite  Test o                          1  7 

I 

physical  and  mental  tests  upon  their  entrance  into  the  industrial 
world :  Height,  Weight,  Visual  Acuity,  Auditory  Acuity,  Vital 
Capacity,  Strength  of  Hand,  Steadiness  of  Hand,  Tapping,  Card 
Sorting,  Cancellation  Test,  Memory  (Digits),  Substitution, 
Completion  of  Sentences,  Association  by  Opposites,  and  Puzzle 
Box  Test. 

A  year  later  six  hundred  and  seventy-nine  of  these  boys  and 
girls  were  re-tested. 

In  giving  the  opposites  test  one  of  the  eight  lists  of  twenty 
words  printed  one  under  another  was  presented  to  the  subject 
who  was  requested  to  write  beside  each  word  another  word  op- 
posite in  meaning.  The  time  was  recorded  for  the  total  list  but 
the  scores  were  based  on  the  percentage  of  accuracy  alone.  Cred- 
its of  one,  one-half,  or  zero  were  assigned  to  the  responses  given. 
Misspelled  words  were  given  full  credit  but  adjectives  written  in 
place  of  adverbs  received  only  half  credit. 

A  positive  correlation  was  obtained  between  the  school  grade 
completed  at  fourteen  years  of  age  and  the  ranking  in  every  one 
of  the  mental  tests.     The  general  order  is  as  follows :     Memory, 
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Association  by  Opposites,  Sentence  Test,  Substitution  Test,  Can- 
cellation, and  Puzzle  Box. 

The  public  school  children  were  found  to  be  superior  to  the 
parochial  school  group,  and  their  superiority  was  most  decided  in 
opposites  and  in  the  puzzle  box  tests,  the  two  tests  which,  accord- 
ing to  the  investigators,  are  farthest  removed  from  a  relationship 
with  school  drill. 

''There  is  a  marked  and  consistent  positive  correlation  with 
school  grade  in  this  test  (opposites)  for  both  sexes  and  at  both 
ages.  The  differences  from  grade  to  grade  are  so  large  and  so 
consistent  that  their  significance  cannot  be  questioned."* 

While  director  of  the  Department  of  Psychology  at  the  New 
York  State  Reformatory  for  Women,  at  Bedford  Hills,  Dr. 
WeidensalP  began  a  series  of  experiments  upon  the  women  de- 
tained in  that  institution.  It  was  hoped  that  a  number  of  tests 
might  be  found  which  would  prove  prophetic  of  the  convicted 
woman's  reformability  and  would  thus  eliminate  such  cases 
which,  because  of  inability  to  learn,  consumed  a  disproportionate 
amount  of  time. 

The  major  portion  of  the  monograph  deals  with  the  records 
of  the  criminal  woman  as  compared  wath  the  norms  of  normal 
working  girls  fourteen  and  fifteen  years  of  age.  The  norms  were 
being  formulated  at  the  time  by  Dr.  Woolley*^  in  her  work  as 
director  of  the  Bureau  of  Vocational  Guidance,  Cincinnati,  Ohio. 
Since  comparisons  were  to  be  made,  the  tests  which  were  being 
used  by  Dr.  Woolley  were  adopted  by  Dr.  Weidensall  and  given 
in  approximately  the  same  manner. 

The  women  tested  w^ere  admitted  between  the  first  of  January 
and  the  end  of  October,  19 13.  Two  hundred  and  eight  were 
committed  to  Bedford  during  that  period,  but  of  that  number 
only  a  hundred  were  tested.  Because  of  lack  of  facility  in  the 
English  language  of  twelve  foreigners,  the  percentile  tables  and 
curves  are  based  on  eighty-eight  records.     All  tests  were  given 

*  Woolley,  Helen  Thompson,  and  Fischer,  Charlotte  Rust,  Op.  cit.,  222. 
5  Weidensall,  Jean,  "The  Mentality  of  the   Criminal  Woman,"   Baltimore, 
Warwick  &  York,  1916,  3-266. 

^  Woolley,  Helen  Thompson,  and  Fischer,  Charlotte  Rust,  Op.  cit. 
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individually  and  during  the  first  two  weeks  of  confinement  while 
the  subject  was  still  in  quarantine. 

The  method  followed  in  connection  with  the  opposites  test 
was  to  present  a  list  of  twenty  words  and  to  record  the  total  time 
required  for  the  subject  to  write  the  opposites.  Results  were 
based  on  the  percent  of  accuracy,  as  it  was  found  that  the  cor- 
relation between  rank  in  time  and  accuracy  in  this  test  was  as 
high  as  .83,  P.  E.  .029. 

All  identical  accuracy  scores  were  arranged  in  point  of  time 
of  performance  and  the  rankings  correlated  with  the  native 
ability  of  the  group  as  estimated  by  the  director  of  the  Industrial 
School  of  the  Reformatory  after  she  had  worked  with  these 
women  for  from  eighteen  months  to  two  years.  This  correlation 
was  -j-  .79,  P.  E.  .026,  and  was  obtained  by  the  formula 

6  2  d' 

Y=  I 

n  (n-  —  I) 
This  correlation,  which  was  higher  than  that  for  any  other  test, 
would  probably  have  been  even  higher  had  it  not  been  necessary 
for  the  director  to  base  her  judgment  of  ten  of  the  women  on  the 
reports  of  matrons.  It  was  the  difference  in  rank  accorded  these 
ten  which  was  responsible  for  some  of  the  largest  variations 
from  the  rank  of  the  test. 

It  is  interesting  to  note  that  of  the  Bedford  eighty-eight  but 
thirty-nine  percent  attain  or  surpass  the  median  record  of  the 
working  girl  of  fifteen.  Dr.  Weidensall  feels  that  the  easy  op- 
posites test  proved  the  most  reliable  of  all  tests  given,  for  clinical 
purposes. 

In  1912  Dr.  Benjamin  R.  Simpson^  selected  two  groups  as 
widely  difTerent  in  intellectual  status  as  possible,  the  one  repres- 
ented by  seventeen  professors  and  advanced  students  in  Columbia 
University,  the  other  by  inmates  of  charitable  institutions,  with 
the  exception  of  two  who  were  recognized  by  their  associates 
as  being  dull.  The  fifteen  tests,  which  were  given,  were  admin- 
istered individually  and  in  the  same  order. 

The  hard  opposites  test  not  only  separated  the  two  groups  com- 

"  Simpson,    Benjamin    R.,   "Correlations   of    Mental   Abilities,"    Col.    Univ. 
Cont.  Ed.,  1912,  No.  53. 
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pletely  but  correlated  with  the  general  intelligence  of  the  good 
group,  as  estimated  by  the  members  which  composed  it,  more 
highly  than  any  of  the  other  tests.  The  individuals  of  the  good 
group  were  rated  in  order  of  merit  for  general  intelligence,  each 
by  the  rest  of  the  group,  four  years  after  the  tests  were  given. 
Two  rankings  made  by  the  experimenter  a  month  apart  were 
included.  The  judgments,  correlated  with  the  various  tests,  are 
as  follows : 

TABLE  III 

Simpson's  Correlations  of  the  Estimated  Intelligence  of  his  Superior 

Group  with  the  Results  of  Eleven  Tests 

Estimated  Intelligence  and  Hard  Opposites 96 


Ebbinghaus  Completion  Test. 

Memory  for  Words   

Memory  of  Passages  

Easy  Opposites    

Adding    

Learning   Pairs    

Completing  Words    

"A"  Test    

Geometrical    Forms    

Drawing  Lengths    


89 
93 
35 
82 
72 

34 

00* 

21 

07 
20 


*This  coefficient,  according  to  Dr.  Simpson,  is  not  to  be  considered  reliable, 
since  the  reliability  coefficients  of  the  Completing  Words  test  in  the  Good 
group  is  only  .27. 

On  account  of  the  high  correlation  between  the  hard  opposites 
and  the  Ebbinghaus  Completion  Test,  Dr.  Simpson  feels  that  they 
test  the  same  mental  function,  namely  selective  thinking. 

Bronner®  has  endeavored  to  determine  the  intellectual  status 
of  the  delinquent  girl  as  compared  with  the  intelligence  of  several 
other  groups  engaged  in  occupations  and  pursuits  which,  of  neces- 
sity, required  varied  degrees  of  education  and  ability.  Thirty 
girls  living  at  Waverly  House,  a  detention  home  maintained  by 
the  New  York  Probation  Association,  composed  the  delinquent 
group.  The  college  group  contained  thirty-six  girls,  all  mem- 
bers of  the  freshmen  and  sophomore  classes  of  Barnard  and 

s  Bronner,  Augusta  P.,  "A  Comparative  Study  of  the  Intelligence  of  De- 
linquent Girls,"  Col.  Univ.  Cont.  Educ,  1914,  No.  (&. 
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Teachers'  College.  Thirty-four  Jewish  girls,  who  spent  their 
evenings  at  the  University  Settlement  or  the  Harlem  Branch  of 
the  Y.  W.  C.  A.,  engaging  in  educational  pursuits,  formed  the 
evening  class  group.  The  fourth  group  was  composed  of  twenty- 
nine  girls  who  had  never  engaged  in  any  wage-earning  occupa- 
tion except  domestic  service.  None  of  these  girls  was  pursuing 
studies  whereby  she  hoped  to  prepare  herself  for  a  different  oc- 
cupation nor  had  she  been  guilty  of  offenses  which  had  brought 
her  in  conflict  with  the  law. 

In  addition  to  an  ethical  discrimination  test,  those  tests  em- 
ployed comprise  the  Easy  Opposites,  the  Hard  Opposites,  the 
Memory  of  Words,  the  Memory  of  Passages,  and  the  Ebbinghaus 
Completion  test.  These  tests  were  the  same  as  those  used  by  Dr. 
Benjamin  Simpson  in  his  study  of  Correlations  of  Mental  Abili- 
ties. In  the  following  table,  which  is  a  reorganization  of  five 
tables  presented  by  Bronner,  D  represents  the  delinquent  group, 
C  the  college  group,  E  the  evening  class  group,  and  S  the  domestic 
service  group.     These  data  show,  as  Bronner  states,  that  the 

TABLE  IV 
Comparisons  of  the  Four  Groups 

Easy   Opposites 

%  of  D  reaching  the  25  percentile  of  the  C  group 3-3% 

%  of  E  reaching  the  25  percentile  of  the  C  group 15.0% 

%  of  S  reaching  the  25  percentile  of  the  C  group 7-0% 

Hard  Opposites 

%  of  D  reaching  the  25  percentile  of  the  C  group 0.0% 

%  of  E  reaching  the  25  percentile  of  the  C  group 12.0% 

%  of  S  reaching  the  25  percentile  of  the  C  group 0.0% 

Memory  of  Unrelated  Words 

%  of  D  reaching  the  25  percentile  of  the  C  group 20.0% 

%  of  E  reaching  the  25  percentile  of  the  C  group 59.0% 

%  of  S  reaching  the  25  percentile  of  the  C  group 7-0% 

Memory  of  Passages 

%  of  D  reaching  the  25  percentile  of  the  C  group 10.0% 

%  of  E  reaching  the  25  percentile  of  the  C  group 15.0% 

%  of  S  reaching  the  25  percentile  of  the  C  group 3.0% 

Ebbinghaus  Completion  Test 

%  of  D  reaching  the  25  percentile  of  the  C  group 6.6% 

%  of  E  reaching  the  25  percentile  of  the  C  group 9.0% 

%  of  S  reaching  the  25  percentile  of  the  C  group 3.0% 
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college  girls  excel  in  all  the  tests.  The  delinquent  girl  is  quite  as 
capable  as  the  domestic  service  girl,  while  both  are  surpassed  by 
the  girls  attending  evening  classes. 

The  superiority  of  the  evening  class  girls  over  the  other  two 
groups  displayed  in  each  test,  cannot  be  attributed  to  more  favor- 
able educational  advantages  for  none  of  the  members  had  attended 
high  school,  whereas  of  the  delinquent  group,  four  had  reached 
the  eighth  grade,  two  the  first  year  of  high  school,  two  the  second 
year,  one  was  graduated,  and  one  had  attended  a  normal  school. 

The  Hard  Opposites  Test  separated  the  college  group  from  the 
others  almost  entirely.  Next  in  order  comes  the  Ebbinghaus 
Completion  Test,  with  the  Easy  Opposites  ranking  third.  It  is 
not  without  interest  that  Dr.  Simpson  found  the  Hard  Opposites 
separated  his  good  group  entirely  from  his  poor  group  and  the 
Easy  Opposites  in  this  respect  surpassed  the  Ebbinghaus  Test. 

Let  us  now  examine  the  record  of  the  Opposites  Test  when 
applied  to  the  highly  selected  group,  represented  by  the  college 
student.  In  1914  Dr.  Kitson"  began  a  series  of  tests  upon  the 
freshmen  in  the  College  of  Commerce  and  Administration  of  the 
University  of  Chicago.  One  of  the  sixteen  tests  used  was  an 
opposites  test.  Two  lists  of  twenty  words  each  were  presented 
and  the  time  recorded  for  the  subject  to  call  the  opposites  of  each 
list.  Five  points  were  deducted  for  each  wrong  word  or  for 
failure  to  respond  within  fifteen  seconds,  from  the  one  hundred 
points  allowed  each  list.  The  final  score  for  the  individual  was 
obtained  by  dividing  the  time  score  by  the  accuracy  score. 

The  stimuli  were  of  such  little  difficulty  that  they  measured  for 
the  most  part  merely  speed  of  association.  Almost  half  of  the 
subjects  obtained  an  accuracy  score  of  one  hundred  in  each  list. 
Of  forty  students,  thirty-one  scored  one  hundred  in  accuracy  in 
the  easy  list,  and  twenty-one  scored  a  hundred  in  the  hard  list. 

In  his  results  Kitson  has  included  the  following  table  of  cor- 
relations. 

Of  the  fifteen  tests  used  by  Kitson,  by  computing  the  multiple 
and    the    partial    correlations    and    the    regression    coefficients, 

9  Kitson,  H.  D..  "The  Scientific  Study  of  College  Students,"  Psychol. 
Monog..  igi',  22,. 
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TABLE  V 

Correlations  of  Standings  in  Each  Test  with  Standings  in  the  Net  Score 
(Method  of  Rank  Differences) 


Logical   Material    Seen    (Deferred)    . 

Opposites    

Hard  Directions    (Printed)    

Obj  ects  Seen   

Loss  in  Logical  Material  Seen  

Logical  Material  Heard  (Deferred)    . 

Word   Building    

Loss  in  Logical  Material  Heard  

Sentence   Building    

Constant  Increment  

Business    Ingenuity    

Logical  Material  Seen  (Immediate)    . 

Numbers  Heard   

Hard  Directions   (Oral)    

Logical  Material  Heard    (Immediate) 
Number   Checking   


ations 

P.  E. 

.60 

.07 

•53 

.08 

.49 

.08 

.48 

.08 

•  47 

.09 

•45 

.09 

•45 

.09 

•43 

.09 

•  42 

.09 

•  38 

.10 

•  35 

.10 

.29 

.10 

.27 

.10 

•  23 

.11 

.23 

.11 

.18 

.11 

Rosenow^"  concludes  that  five  of  the  tests  carry  all  the  meaning 
with  reference  to  school  marks  and  hence  all  diagnostic  value. 
In  the  table  below  the  w^riter  has  rearranged  the  results  of 
Rosenow's  investigation.  From  it  one  learns  that  the  probability 
is  1300  :  I  that  the  Logical  Memory  is  significant. 

TABLE  VI 
Rearrangement  of  the  Results  of  Rosenow's   Investigation 
Probability  is  1300  :  i  that  the  Logical  Memory  Test  is  Significant. 
"  "        25  :  I  that  the  Constant  Increment  Test  is  Significant. 

"  "        30  :  I  that  the  Sentence  Building  Test  is  Significant. 

"  "        23  :  I  that  Auditory  Presentation  is  Superior  to  Visual. 

"  "       140  :  I  that  the  Loss  or  Gain  in  Logical  Auditory  Memory 

is   significant. 
"  "      that  the  Hard  Directions  Test  has  Negative  Significance. 

"  "      that  the  Objects  Seen  Test  has  Negative  Significance. 

The  remaining  tests,  including  the  Opposites,  have  no  diag- 
nostic value. 

In  view  of  the  fact  that  the  stimuli  used  in  the  Opposites  Test 
were  the  Woodworth  and  Wells  standardized  list,  Rosenow's  con- 

10  Rosenow,  Curt,  "The  Analysis  of  Mental  Functions,"  Psychol.  Monog., 
1917,  24. 
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elusions  are  not  surprising.    The  words  were  of  so  little  difficulty 
that  they  tested  only  speed  of  association. 

King  and  M'Crory"  in  testing  two  hundred  and  seventy-six 
women  and  two  hundred  and  sixty-eight  men  in  the  freshmen 
class  at  the  University  of  Iowa,  applied  seven  tests,  including 
both  easy  and  hard  opposites.  The  following  table  indicates  the 
correlations  which  were  obtained  between  the  Opposites  and 
other  tests  as  well  as  university  grades.  The  Opposites  Test, 
they  find,  correlates  most  highly  of  any  of  the  tests  with  the 
university  marks  of  both  men  and  women. 

TABLE  VII 

Correlations  of  Opposites  with  Other  Tests  and  University  Grades 

Correlations 
Women      Men 

Opposites  and  Completion  31  .79 

"    Arithmetic   (Speed)    03        Neg. 

"    Arithmetic  (Accuracy)    01        Neg. 

"    Analogies   52  .77 

"     Information    24  .56 

"    Visual  Imagery   07  .56 

"     Logical   Memory    $2  .38 

"    Test  Average   51  .88 

"    University   Grades    45  .84 

Dr.  King^"  is  of  the  opinion  that  the  Opposites  Test,  if  thor- 
oughly standardized  and  used  in  conjunction,  with  other  tests, 
will  yield  results  of  great  importance. 

In  the  Spring  of  1916,  while  the  writer  was  yet  engaged  in  the 
selection  of  appropriate  stimuli  for  the  present  investigation,  an 
opportunity^^  presented  itself  of  correlating  the  grades  of  seventy- 
three  students  in  psychology  with  the  results  secured  by  the  use 
of  the  Opposites  Test.  An  effort  had  been  made  to  keep  the  two 
lists  composed  of  one  hundred  and  fifty  words  each,  as  nearly 

11  King,  Irving,  and  M'Crory,  J.  L.,  "Freshmen  Tests  at  the  State  Univer- 
sity of  Iowa,"  Jour.  Educ.  Psychol.,  1918,  IX,  32-46. 

12  King,  Irving,  and  Gold,  Hugo,  "A  Tentative  Standardization  of  Certain 
Opposites  Tests,"  Jour.  Educ.  Psychol.,  1916,  VII,  459-482. 

13  Dr.  Edward  K.  Strong,  at  that  time  Professor  of  Psychology  at  George 
Peabody  College  for  Teachers,  kindly  furnished  these  data.  The  grades 
were  compiled  from  the  records  of  fourteen  tests  given  during  the  quarter. 
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equal  as  possible.  The  two  lists  are  given  below.  The  words 
are  arranged  in  random  order.  Those  words  which  had  not  been 
used  by  previous  investigators  are  in  italics. 

List  I. — Defective,  Late,  Sinful,  Easy,  Hilly,  Superior,  Girl, 
White,  Cool,  Large,  Evil,  Queen,  Deep,  True,  Public,  Sink, 
Future,  Adroit,  Dangerous,  Day,  Ugly,  Quick,  Poor,  Diligent, 
Wicked,  Round,  Ceiling,  Broken,  Gentle,  Vague,  Brief,  Ani- 
mated, Slovenly,  Dim,  Out,  Rude,  Lazy,  Injurious,  Conservative, 
Wet,  Asleep,  Stingy,  Fertile,  Wise,  Calm,  Tardy,  Hinder,  Re- 
spect, Big,  Gain,  Great,  Profit,  Young,  Few,  Summer,  Above, 
Glad,  Masculine,  Remember,  Off,  Beginning,  Love,  Straight, 
War,  Joy,  Naked,  Pride,  Apart,  Brave,  Noisy,  Fickle,  Create, 
Wild,  Despondent,  Frequently,  Timid,  Hollow,  Belief,  Bad,  Up, 
Sick,  Empty,  Strong,  Inside,  Front,  After,  Broad,  Sharp,  Sweet, 
Succeed,  Add,  Happy,  Raise,  Aristocratic,  East,  Short,  Thick, 
Result,  Rare,  Stale,  North,  Hostile,  Laugh,  Obnoxious,  Ex- 
pensive, Near,  Join,  Hot,  Forcible,  Preserve,  Strict,  Handsome, 
Friend,  Miser,  Exciting,  Rough,  Brother,  Light,  Careful,  Push, 
Haughty,  Impoverish,  Busy,  Much,  Graceful,  Ocean,  Precise, 
Barbarous,  Ignorant,  Reckless,  Odd,  Victorious,  Repulsion,  Per- 
mit, Positive,  Pessimistic,  Extravagant,  Durable,  Analytical,  Par- 
simony, Orthodoxy,  Acute,  Exoteric,  Antonym,  Dorsal,  Longi- 
tude, Divide,  Infinity,  Dynamic,  Posterior. 

List  II. — Gay,  Foolish,  Drop,  Giving,  Cloudy,  Blunt,  Beauti- 
ful, Backwards,  Well,  Top,  Success,  Soft,  New,  Refined,  Weary, 
Spend,  Break,  Male,  Country,  Dark,  Weak,  Black,  Disastrous, 
Rigid,  Elation,  Hindrance,  Savage,  Degrade,  Ripe,  Shaky,  Sepa- 
rate, Liquid,  Sell,  Honest.  Difficult,  Dirty,  Wrong,  Winter,  Help- 
less, Obscure,  Expand,  Insignificant,  Sleepy,  Sad,  Little,  Enemy, 
Open,  Yours,  Yes,  Conservative,  Soothing,  Doubtful,  Sacred, 
Sure,  Reveal,  Stupid,  Motion,  Sickly,  Slowness,  Outside,  Same, 
Cowardly,  Float,  Foreign,  Strength,  Sane,  Level,  Simple,  Many, 
Lost,  Something,  Sour,  Enrage,  Serious,  Long,  In,  Take,  Tight, 
Prompt,  Patient,  Permanent,  Genuine,  Morning,  Smooth,  Heavy, 
Full,  Grand,  Humility,  Tall,  Over,  First,  Strife,  Follower,  Hold, 
Proficient,  Vertical,  Shallow,  Absent,  Rapid,  Rich,  Purity,  Lo- 
quacious, Imaginary,   Silly,  Increase,  Wider,  Nowhere,  Upper, 
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Woman,  Generous,  Careless,  Scarce,  Height,  Always,  Wife,  Best, 
Below,  Thin,  High,  Early,  Suave,  Lack,  Advance,  Harmonious, 
Bless,  Both,  Cruel,  Cheap,  Ancient,  Less,  Forget,  Come,  Slow, 
Good,  Negative,  Optimistic,  Economical,  Perishable,  Synthetical, 
Prodigality,  Heterodoxy,  Chronic,  Esoteric,  Synonym,  Ventral, 
Latitude,  Midtiply,  Zero,  Static,  Anterior. 

The  test  was  given  individually  and  orally.  The  experimenter 
read  the  stimulus  and  recorded  the  response  of  the  subjects  as 
well  as  his  reaction  time.  The  accuracy  score  in  percentage  and 
the  time  score  arrived  at  by  computing  the  median  time  of  all 
reactions  were  tabulated  for  each  student. 

In  grading  these  papers,  credits  of  one,  one-half,  or  zero  were 
assigned  to  the  responses.  What  credit  a  word  deserved  was 
determined  solely  by  the  Avriter,  as  she  was  not  yet  ready  to 
standardize  the  responses  for  so  bulky  a  list. 

In  order  to  allow  equal  credit  for  time  and  accuracy  the  follow- 
ing formula  was  used : 

Dt  Da 

Individual  score  = 1 in  which 

OT  OA 


2 
Da  =  the  deviation  of  the  individual  accuracy  scores  from 

the  accuracy  scores  of  the  group. 
Dt    =  the  deviation  of  the  individual  time  score  from  the 
time  score  of  the  group.    Since  signs  were  regarded, 
where  an  individual  had  a  lower  time  score  than  the 
group  his  deviation  was  positive. 
QA=  the  quartile  of  the  accuracy  scores  of  the  group. 
OT=  the  quartile  of  the  time  scores  of  the  group. 
The  coefficient  of  correlation  obtained  by  the  method  of  rank 
differences  was  .54,  P.   E.  =  .08.     Speed  and  accuracy  w^ere 
likewise  correlated,  giving  a  coefficient  of  .33,  P.  E.  =  .10.    This 
coefficient  would  hardly  justify  the  statement  that,  in  so  far  as 
this  particular  test  is  concerned,  either  accuracy  or  speed  could  be 
disregaded  without  affecting  the  rankings  in  the  test. 

In  Figure  I  is  shown  the  relative  positions  of  the  several  mem- 
bers of  the  class  in  psychology  and  in  the  Opposites  Test.     The 
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vertical  line  indicates  the  rank  in  the  test,  the  horizontal  line,  the 
rank  in  psychology.  A  glance  at  the  figure  will  show  that  on  the 
whole  those  good  in  the  test  were  good  in  psychology.  The 
greatest  exception  is  to  be  found  in  the  position  of  the  student 
indicated  by  the  figure  5  in  the  graph.  This  woman  of  splendid 
ability  was  permitted  by  her  physician  to  remain  in  school  only 
on  the  condition  that  she  would  do  the  least  possible  amount  of 
work  to  secure  credit  for  the  course  in  which  she  was  enrolled. 
Case  4,  a  widow,  had  been  out  of  school  and  school  work  for  a 
number  of  years.  The  increased  cost  of  living  had  forced  her 
to  return.  Case  3  had  registered  in  psychology  because  it  was 
one  of  the  required  courses. 
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The  relations  between  rank  based  on  psychology  grades  and  rank  based 
on  the  Opposites  Test  score  are  shown  here.  Each  circle  indicates  the  posi- 
tion of  one  of  the  seventy-three  college  students.  Units  on  the  vertical  line 
denote  rank  in  the  test;  horizontal  units  are  ranks  in  psychology. 
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On  the  other  hand,  it  seems,  one  could  predict  with  less  cer- 
tainty the  standing  in  the  class  of  those  students  who  are  some- 
what poor  in  the  test.  This  may  be  due  in  part  to  the  inadequacy 
of  the  test  itself,  in  part  to  the  inequality  of  interest  in  psy- 
chology. Cases  I  and  2  are  typical  of  those  students  whose  class 
work  represents  their  best  effort.  These  cases  are  selected  be- 
cause they  are  the  only  ones  falling  within  the  writer's  experience. 
A  careful  analysis  of  the  individual  records,  if  possible,  might 
reveal  results  of  significance. 

While  the  results  indicate  that  the  test  as  used  by  the  different 
investigators  reveals  some  sort  of  native  ability,  just  what  this 
ability  is  and  to  what  extent  it  is  revealed  remains  to  be  seen. 
Certainly,  results  of  various  experimenters  up  to  date  are  not 
comparable,  nor  will  they  be  until  identically  the  same  test  is 
given  with  the  same  method  of  presenting  and  scoring. 


III. 

Previous  Efforts  toward  Standardization 

The  first  attempt  to  meet  this  need  of  standardization  was 
made  by  Woodworth  and  Wells^  in  1911,  After  testing  six  sub- 
jects with  a  long  list  of  words,  forty  words  were  selected,  which 
in  turn  were  tried  with  forty  other  subjects.  A  few  more  words 
were  substituted  from  tests  of  a  few  subjects,  then  the  revised  col- 
lection was  tried  with  thirteen  fresh  subjects  (all  college  and 
graduate  students)  and  a  few  more  minor  corrections  introduced. 

In  the  experiment  the  four  lists  of  ten  stimulus  words  were 
presented  visually  but  the  time  of  the  single  reaction  as  well  as 
the  total  time  was  roughly  taken. 

An  attempt  was  made  to  have  the  halves  of  the  list  of  equal 
difficulty,  and  since  it  was  found  impossible  to  prepare  a  list  of 
twenty  stimuli  of  equal  difficulty,  the  words  were  combined  in 
pairs,  "so  that  pairs  should  be  of  equal  difficulty,  as  judged  by 
the  sum  of  the  reaction  times  to  the  two  members  of  each  pair." 
One  pair,  for  instance,  consisted  of  the  hardest  and  the  easiest 
word  in  the  list  and  another  pair  of  two  words  of  medium  dif- 
ficulty. 

In  the  event  that  the  test  might  be  given  with  a  time  limit,  the 
authors  have  placed  the  words  of  medium  difficulty  in  the  list 
where  most  of  the  subjects  will  be  stopped,  namely,  from  about 
the  eighth  to  the  sixteenth  word.  "If  then  the  time  limit  is  so 
chosen  that  the  great  majority  of  subjects  shall  be  stopped  in 
this  list,  the  separate  words  may,  without  much  error  on  the  aver- 
age, be  counted  as  equal  units." 

The  test  as  given  by  Woodworth  and  Wells  is  scored  only  on 
the  basis  of  time.  The  words  were  so  selected  that  none  of  the 
subjects  could  fail  to  have  a  perfect  accuracy  score.  As  such,  it 
is,  as  the  authors  intended,  merely  a  test  of  the  speed  of  associa- 
tion. 

iWoodworth,  R.  S.,  and  Wells,  F.  L.,  "Association  Tests,"  Psychol. 
Monog.,  191 1,  XIII. 
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In  1916  a  further  attempt  was  made  by  King  and  Gold'  to 
standardize  the  test.  The  four  Hsts  of  twenty  easy  words  and  the 
four  Hsts  of  twenty  hard  words,  as  used  by  Simpson,^  were  pre- 
sented individually  to  nine  faculty  members,  twenty-three  gradu- 
are  students,  forty-seven  seniors,  and  twenty-one  juniors  of  the 
Departments  of  Education  and  Psychology  in  the  University  of 
Iowa. 

The  subjects  were  instructed  to  take  one  list  at  a  time,  go  down 
the  list  and  name  orally  the  opposite  of  each  word  in  rapid  suc- 
cession and  to  avoid  wherever  possible  prefixing  a  syllable  to  the 
stimulus  word.  The  time  was  recorded  for  each  list  separately 
and  the  responses  were  taken  down  in  shorthand. 

The  easy  and  hard  opposites  were  scored  differently.  In  the 
case  of  the  easy  opposites  a  credit  of  one,  one-half,  or  zero  was 
given  to  responses,  and  the  penalty  for  an  omitted  word  was  four 
seconds.  To  the  various  responses  to  the  hard  opposites,  credits 
of  one,  two-thirds,  one-third,  and  zero  were  assigned,  and  an 
omitted  word  or  incorrect  word  was  penalized  eighteen  seconds. 
Regardless  of  the  fact  that  the  words  were  of  varying  difficulty, 
the  penalty  remained  the  same,  namely,  four  seconds  for  easy 
words  and  eighteen  seconds  for  hard  words. 

On  the  basis  of  these  results  (one  hundred  records  for  each 
word)  the  percent  of  failures  for  each  word  was  computed.  For 
each  stimulus  is  recorded  the  value  in  term's  of  accuracy  per- 
centage as  w^ell  as  the  acceptable  responses  and  the  frequency  of 
each. 

In  the  work  of  Harry  A.  Greene*  we  find  the  first  attempt  to 
assign  to  each  word  a  point  value  based  on  its  relative  difficulty. 
Greene  presented  the  stimuli  used  by  King  and  Gold^  to  two 
groups  of  freshmen  in  the  University  of  Iowa  in  the  fall  of  1916- 
191 7.  Nine  hundred  and  ninety  individuals  were  tested  by  the 
first  half  of  the  list  and  seven  hundred  and  ten  by  the  second 

-  King,  Irving,  and  Gold,  Hugo,  "A  Tentative  Standardization  of  Certain 
Opposites  Tests,"  Jour.  Educ.  Psychol.,  1916,  VII,  459-482. 

3  Simpson,  Benjamin  R.,  Op.  cit. 

•*  Greene,  Harry  A.,  "A  Standardization  of  Certain  Opposites  Tests,"  Jour. 
Educ.  Psychol.,  1918,  IX,  559-566. 

5  King,  Irving,  and  Gold,  Hugo,  Op.  cit. 
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half.  The  words  were  graded  by  the  King  and  Gold  standardiza- 
tion with  the  exception  that  words  given  two-thirds  credit  by  them 
were  given  one-half  credit  and  those  receiving  one-third  credit 
were  scored  wrong. 

The  percentage  of  failures  was  then  determined  by  allowing 
a  value  of  one  unit  for  correct  responses  and  one-half  for  half- 
correct  responses  and  subtracting  this  total  from  nine  hundred 
and  ninety.  In  the  case  of  those  words  from  the  last  half  of  the 
list  for  which  there  were  only  seven  hundred  and  ten  responses 
available,  these  totals  were  increased  in  the  proportion  of  nine 
hundred  and  ninety  to  seven  hundred  and  ten,  thus  making  all 
the  words  in  the  list  comparable. 

By  reading  directly  from  the  table  based  upon  the  area  of  the 
probability  curve  and  assuming  that  the  base  line  is  broken  ar- 
bitrarily at  -}-3  sigma,  the  percentage  scores  were  changed  into 
percentile  values.  These  values  were  then  totaled  and  each  value 
in  turn  divided  by  the  total,  thus  converting  the  percentile  values 
into  relative  point  values.  The  points  were  based  on  accuracy 
alone,  no  account  being  taken  of  time. 


IV. 

The  Selection  of  Stimuli  for  the 
Present  Investigation 

At  the  very  outset,  the  writer  was  confronted  with  the  problem 
of  the  method  in  which  the  test  was  to  be  presented.  Chiefly 
because  of  the  opportunity  it  afforded  of  studying  each  word 
individually,  the  oral  method  was  chosen.  Each  word  thus  be- 
came a  unit.  The  list  could  be  lengthened  or  shortened  at  will 
without  influencing  to  any  appreciable  extent  the  words  retained. 
Let  it  be  held  in  mind  that  at  this  point  our  interest  was  two-fold, 
namely,  the  selection  of  suitable  stimuli  and  the  computation  of 
the  value  in  points  to  be  assigned  to  each  word. 

To  have  given  the  test  in  groups  would  have  simplified  the 
scoring  and  would  have  dispensed  with  much  effort,  but  the  at- 
tendent  disadvantages  made  it  necessary  to  abandon  this  method 
as  a  possible  choice. 

In  the  first  place,  it  was  intended  that  the  test  should  be  more 
difficult  than  previous  tests  of  this  character.  This  meant  the 
substitution  of  entirely  new  stimuli,  stimuli  to  be  acquired  only 
by  the  slow  process  of  trial  and  error.  Furthermore  it  was  en- 
cumbent upon  the  writer,  due  to  the  small  number  of  subjects 
at  her  disposal,  to  avoid  a  method  which  would  involve  discard- 
ing the  entire  record  of  a  subject  because  of  the  use  of  one  or 
more  undesirable  stimuli.  As  given,  each  word  was  a  problem 
apart  from  every  other  word  and  only  the  records  of  those  words 
not  in  the  list  as  finally  decided  upon,  were  thrown  into  the 
discard. 

If  the  group  method  is  to  be  employed,  a  decision  must  be 
made  as  to  whether  time,  accuracy,  or  a  combination  of  both  will 
be  considered  a  measure  of  the  difficulty  of  the  word.  Time 
considered  alone,  the  test  becomes  one  largely  of  speed  of  asso- 
ciation. If  the  test  is  to  be  given  with  a  time  limit,  the  individual 
responses  lose  their  identity  as  units  and  the  percentage  of  failures 
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per  stimulus  will  vary  with  the  time  allowed.  For  instance,  if 
the  time  limit  is  reduced  from  six  to  three  minutes,  the  percentage 
of  failures  will  vary  considerably,  particularly  in  the  case  of 
the  more  difficult  words  which  should  be  placed  at  the  end  of  the 
list.  Thus  one  can  see  how  a  shortened  time  limit  might  give 
identical  accuracy  scores  with  words  of  unequal  difficulty. 

To  illustrate  the  effect  of  varying  the  time  limit  not  only  upon 
the  percentage  of  failures  but  also  upon  the  nature  of  the  re- 
sponses themselves,  let  us  consider  the  responses  to  the  word 
"sacred"  given,  first,  by  a  group  of  fifty-six  students  tested  orally 
and  individually,  allowing  ten  seconds  for  the  response,  second, 
by  another  group  of  fifty-six  students  similarly  tested,  and  third, 
by  a  group  of  fifty-six  who  were  allowed  only  six  minutes  to 
write  the  opposites  to  the  list  of  sixty-eight  words  eventually  re- 
tained. In  the  table  below  is  shown  the  number  of  failures  and 
the  exact  responses  given  by  the  three  groups  of  fifty-six.  Notice 
the  greater  number  of  failures  in  Group  III  and  also  the  homo- 
geneity of  responses.  This  may  be  explained  by  the  fact  that  the 
word  is  not  reached  in  the  list  or  that  uncertain  words  are  passed 
up  for  those  in  which  the  subject  feels  confident  of  success. 

On  the  other  hand  to  lengthen  the  time  limit  considerably, 
while  it  would  give  more  nearly  correct  accuracy  scores,  would 
be  unfair  to  the  individuals  tested,  if  these  same  records  were  to 
be  used  as  the  basis  for  norms.  With  a  lengthened  time  limit, 
students  would  be  able  to  complete  the  test  at  different  intervals. 
Evidently  of  two  students,  both  of  whom  make  perfect  scores, 
the  one  finishing  the  work  in  the  shorter  period  is  the  more  ef- 
ficient.   But  we  should  have  no  measure  of  this  efficiency. 

To  give  the  test  as  a  group  test  without  taking  into  considera- 
tion the  time  consumed,  would  seem  to  imply  either  that  addi- 
tional time  would  fail  to  increase  the  accuracy  percentage,  or 
that  without  exception  the  more  difficult  a  word,  the  longer  the 
time  required  to  think  of  an  opposite.  The  former  assumption  is 
probably  true  within  certain  limits.  It  is  conceivable  that  a  time 
limit  might  be  selected  beyond  which  there  would  be  no  improve- 
ment, but  it  is  obvious  that  the  time  limit  might  be  so  shortened 
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TABLE  VIII 

Showing  the  Different  Responses  Given  as  an  Opposite  to  the  Word 
"Sacred"  when  the  Time  Limit  is  Changed 

Number  of  Times  Each  Response 
is  given  by  the  Several  Groups 


Different  Responses 
Given  by  the  Groups 


Group  I      Group  II    Group  III 


Common  . .  . 
Secular  . .  . . 
Profane  . . . . 

Sinful     

Worldly  . . . 
Unholy  . . .  . 
Irreverent  . 
Wicked  . . . . 
Sacrilegious 
Unsacred    . . 

Vulgar  

Public    

Mean    

L^ngodliness 
Heathen  . . . 
Sacrilege  . . . 
Obnoxious  . 
Ungodly  . .  . 
Unnoticed  . . 
Idolatrous  . . 
Irreligious    . 

Infidel    

Hypocritical 


2 

I 

I 

O 

5 

0 

2 

7 

2 

4 
4 
I 

0 

2 

I 

9 

2 
2 

I 

3 

2 

2 

I 
I 

3 
8 

I 

2 

7 
5 
3 
I 

2 
0 
0 
0 

0 

I 

0 

0 

I 

0 

0 

I 

0 

0 

I 

0 

0 

0 

o 

o 

0 

0 

0 

0 

0 

0 

0 

0 

0 

0 

Failure  to  Respond 15 


16 


39 


Number  of  Different 
Responses  per  Group. 


16 


16 


that  the  accuracy  scores  would  be  greatly  affected.  In  general 
it  is  true  that  words  with  a  high  accuracy  score  have  a  corre- 
spondingly low  time  score,  but  a  glance  at  Tables  XI  and  XII 
will  show  that  such  is  not  always  the  case,  for  words  with  differ- 
ent accuracy  scores  may  have  identical  time  scores  and  words 
with  identical  accuracy  scores  have  different  time  scores.  How- 
ever, the  correlation  between  the  two  is  undoubtedly  high. 

To  give  the  test  individually  and  record  the  time  for  the  total 
list  has  all  the  disadvantages  in  point  of  labor  and  none  of  the 
advantages  that  accrue  when  the  test  is  given  individually  and 
orallv. 
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Having  determined  the  procedure  to  be  followed,  the  writer 
began  in  the  fall  of  1915  the  task  of  selecting  suitable  stimuli 
for  the  present  investigation.  After  eleven  records  were  obtained 
with  a  list  containing  the  three  hundred  and  twenty-three  words 
used  by  previous  experimenters  and  forty  new  words,  the  list 
was  divided  and  during  the  remainder  of  the  year  two  lists  were 
used  which  consisted  on  the  average  of  about  one  hundred  and 
fifty  words  each.  From  time  to  time  words  found  inadequate 
were  dropped  and  others  substituted  as  they  occurred  to  the 
writer.  Hence  the  inequality  in  the  number  of  subjects  tested 
with  a  given  word.  In  this  manner  one  hundred  and  thirty-nine 
students  at  George  Peabody  College  for  Teachers  were  tested 
during  the  school  year  1915-1916. 

Commonly  misunderstood  words  were  weeded  out  as  were 
those  with  an  accuracy  score  of  one  hundred  percent.  Likewise 
attempts  were  made  to  discard  those  words  which  had  an  oppo- 
site formed  by  adding  the  prefix  "un"  in  frequent  and  reputable 
use.  Meanwhile  original  words  were  subjected  to  the  same  pro- 
cess of  examination  and  elimination. 

The  point  of  interest  was  the  individual  word,  but  as  these 
data  were  to  be  used  in  another  connection  care  was  taken  that 
the  conditions  remain  constant  throughout.  Since  each  test  was 
given  individually,  it  was  an  easy  matter  to  change  the  order  of 
the  stimuli  and  thus  avoid  practice  effects.  The  reaction  time  in 
fifths  of  a  second  was  recorded  with  a  stop-watch  along  with  the 
response.    The  following  directions  were  read  to  each  subject: 

"As  soon  as  I  read  a  word  you  are  to  give  me  the  best  opposite 
you  can  think  of.  For  instance,  if  I  read  'black'  you  are  to  say 
'white.'  Do  not  give  me  phrases,  nor  words  beginning  with  the 
prefix  'non.'  The  word  you  give  must  belong  to  the  same  part 
of  speech  as  the  word  in  the  list.  Your  time  will  be  recorded, 
so  answer  each  word  as  quickly  as  possible.  Under  no  circum- 
stances will  I  allow  you  over  ten  seconds  for  a  word.  Let  me 
show  you  how  long  ten  seconds  really  is.  (Experimenter  illus- 
trates with  stop-watch.)  Remember  in  each  case  to  respond  as 
quickly  as  possible.     Do  you  understand  what  you  are  to  do?" 
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In  the  fall  of  1918^  the  work  was  resumed.  It  was  decided 
expedient  to  use  only  one  list  as  the  number  of  possible  subjects 
to  be  obtained  at  Peabody  was  small.  Two  lists  would  of  course 
cut  the  records  for  each  word  in  two.  Besides  the  second  list 
contained  for  the  most  part  opposites  of  the  first.  Of  the  one 
hundred  words  with  which  the  work  was  begun  in  the  fall,  thirty- 
two  were  dropped,  leaving  the  sixty-eight  which  comprise  the 
list  as  it  now  stands.  These  words,  arranged  in  the  ascending 
order  of  difficulty,  are  given  below.  The  original  words  are 
given  in  italics.  Full,  Negative,  After,  Dim,  Blunt,  Success, 
Pessimistic,  Joy,  Public,  Profit,  Spend,  Always,  Graceful, 
Strength,  Ancient,  Expand,  Barbarous,  Hinder,  Despondent, 
Vague,  Fertile,  Doubtful,  Injurious,  Busy,  Abstract,  Advance, 
Foreign,  Create,  Simple,  Extravagant,  Aristocratic,  Rare,  Dan- 
gerous, Slovenly,  Defective,  Stingy,  Reveal,  Diligent,  Join,  Im- 
poverish, Permanent,  Elation,  Sinful,  Obnoxious,  Conservative, 
Victorious,  Obscure,  Proficient,  Rigid,  Repulsion,  Imaginary, 
Permit,  Orthodoxy,  Analytical,  Extrinsic,  Sacred,  Dynamic,  Lo- 
quacious, Heterogeneous,  Spurious,  Disastrous,  Facility,  Pride, 
Result,  Adroit,  Parsimony,  Suave,  Esoteric. 

One  hundred  and  twelve  subjects  were  tested  with  this  list. 
This  means  that  later  in  determining  the  relative  difficulty  of 
the  stimuli,  the  calculations  are  based  on  at  least  one  hundred 
and  twelve  records  for  each  word,  varying  up  to  one  hundred 
and  eighty-seven  for  others. 

1  This  research  was  conducted  under  the  supervision  of  Dr.  Edward  K. 
Strong  up  to  this  point,  thereafter  under  the  guidance  of  Dr.  Joseph 
Peterson. 


V. 

The  Choice  of  the  Acceptable  Responses 

The  next  step  was  to  standardize  and  evaluate  the  responses. 
In  order  to  facilitate  matters,  beneath  each  stimulus  was  written 
the  long  list  of  words  which  had  been  given  as  opposites  by  those 
subjects  experimented  upon.  The  credit  due  each  of  these  re- 
sponses was  determined  by  five  judges,  including  the  writer. 
They  were  besides  the  writer : 

Miss  Lula  O.  Andrews,  Professor  of  English;  Miss  Mary  Clay 
Hiner,  Instructor  in  English;  Mr.  S.  H.  Phelps,  Instructor  in 
School  Administration,  and  Dr.  Joseph  Peterson,  Professor  of 
Psychology,  all  of  George  Peabody  College. 

Each  judge  was  ignorant  of  the  credit  assigned  any  word  by 
any  other  judge.  The  following  directions,  a  copy  of  which  was 
handed  each  judge,  will  make  the  matter  clear: 

Directions  for  Grading  Responses 

I.  Make  use  of  any  available  source  of  information  such  as 
the  dictionary  or  book  of  antonyms. 

II.  The  responses  are  to  be  graded  with  a  grade  of  "one," 
which  means  an  exact  opposite,  or  "one-half,"  which  means 
only  an  approximate  opposite,  or  "zero,"  which  means  a  failure. 

III.  More  than  one  response  to  a  word  may  be  given  a  credit 
one. 

IV.  Words  belonging  to  a  different  part  of  speech  are  to  be 
graded  "zero,"  also  words  with  the  prefix  "non." 

V.  If  a  word  belongs  to  the  same  part  of  speech  and  is  nearly 
an  opposite,  give  it  "one-half"  credit. 

VI.  Add  any  response  that  may  suggest  itself  to  you  and 
grade  it  as  directed  above. 

The  new  words  suggested  were  in  turn  passed  on  by  the  other 
four  judges.  In  addition,  when  later  the  test  was  converted 
into  a  group  test  and  hundreds  of  records  secured,  a  number  of 
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different  responses,  given  by  those  tested,  were  submitted  to  the 
same  process  of  evahiation. 

The  credits  assigned  each  response  were  then  averaged  and 
if  the  result  was  nearer  "zero"  than  "one-half,"  the  word  re- 
ceived no  credit;  if  nearer  "one-half"  than  "zero"  or  "one"  it 
received  "one-half"  credit;  and  if  nearer  "one"  than  "one-half," 
it  was  given  a  credit  of  "one."  For  instance  the  response 
"young"  in  answer  to  "ancient"  received  credits  as  follows : 


TABLE  IX 

I 

II 

III       IV 

Y 

Total 

Credit 
Aver.  Assigned 

Credits  given 
to  "young" 
by  five   judges. . 

I 

H 

0           0 

V2 

2 

.4            V2 

On  the  following  pages  is  to  be  found  in  Column  II  the  stim- 
ulus word,  in  Column  III  the  responses  which  are  due  a  credit  of 
"one'  (the  full  value  assigned  to  the  word),  and  in  Column  IV 
those  responses  which  receive  only  "one-half"  credit  (half  the 
value  assigned  the  word).  The  value  of  the  word  is  given  in 
points  in  Column  I.  The  manner  in  which  we  arrived  at  these 
values  will  be  discussed  in  the  following  section. 

TABLE  X 

A  List  of  the  Stimuli  with  Assigned  Values  and  Accepted  Responses 


Value 

Stimulus  Word 

Correct  Responses 

Half  Correct  Responses 

I 

Full 

Empty 

Meagre 

I 

Negative 

Affirmative 
Positive 

Assertive 

I 

After 

Before 

Preceding 

Fore 

I 

Dim 

Bright 
Clear 
Distinct 
Luminous 

Light 
Plain 
Vivid 

I 

Blunt 

Keen 
Pointed 
Sharp 
Tactful 

Acute 
Polite 
Sensitive 
Suave 

26 


MARIE  HACKL  MEANS 


TABLE  X— Continued 


Value 

Stimulus  Word 

Correct  Responses 

Half  Correct  Responses 

I 

Success 

Defeat 

Disaster 

Failure 

Downfall 

Loss 

I 

Pessimistic 

Optimistic 

Joyful 

I 

Joy 

Gloom 

Grief 

Misery 

Sadness 

Sorrow 

Depression 

Despondency 

Displeasure 

Pain 

Woe 

2 

Public 

Private 

Domestic 
Personal 
Secluded 
Secret 

2 

Profit 

Loss 

Deficit 
Lose 

2 

Spend 

Earn 

Hoard 

Husband 

Keep 

Save 

Accumulate 

Get 

Hold 

Make 

Receive 

2 

Always 

Xever 

Infrequently 
Seldom 

2 

Graceful 

Awkward 
Clumsy 
Gawky 
Ungainly 

Uncouth 

Ungraceful 

Unsightly 

2 

Strength 

Feebleness 

Frailty 

Weakness 

Delicacy 
Insecurity 

3 

Ancient 

Modern 

New 
Present 
Recent 
Young 

3 

Expand 

Contract 
Decrease 

Compress 
Narrow 

Diminish 

Shrink 

Shrivel 


Barbarous 


Civilized 
Humane 


Chivalrous 

Civil 

Cultured 

Educated 

Gentle 

Kind 

Polite 
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TABLE  X— Continued 


Value 

Stimulus  Word 

Correct  Responses 

Half  Correct  Responses 

3 

Hinder 

Aid 

Advance 

Assist 

Encourage 

Expedite 

Fore 

Forward 

Further 

Facilitate 

Help 

Promote 

3 

Despondent 

Buoyant 

Bright 

Cheerful 

Encouraged 

Ecstatic 

Exhilarated 

Elated 

Gay 

Exuberant 

Hilarious 

Glad 

Joyous 

Happy 

Merry 

Hopeful 

Optimistic 

Joyful 

Sanguine 

Jubilant 

3 

Vague 

Clear 

Apparent 

Definite 

Concise 

Distinct 

Explicit 

Exact 

Plain 

Specific 

3 

Fertile 

Arid 

Fruitless 

Barren 

Impotent 

Poor 

Impoverished 

Sterile 

Infertile 

Unproductive 

Unimaginative 

4 

Doubtful 

Assured 

Apparent 

Certain 

Believable 

Evident 

Clear 

Hopeful 

Confident 

Sure 

Credible 

Known 

Positive 

True 

Truthful 

Unquestioned 

4 

Injurious 

Advantageous 

Harmless 

^ 

Beneficial 

Healthful 

Helpful 

Safe 

Innocuous 

Uninjurious 

Wholesome 

4 

Busy 

Idle 

Unemployed 

Dull 
Inactive 

Unoccupied 

Indolent 
Loafing 

^ 


MARIE  HACKL  MEANS 


TABLE  X— Continued 


Value 

Stimulus  Word 

Correct  Responses 

Half  Correct  Responses 

4 

Abstract 

Concrete 

Specific 

4 

Advance 

Decline 

Degrade 

Recede 

Hinder 

Retard 

Withhold 

Retire 

Retreat 

Retrograde 

Withdraw 

4 

Foreign 

Domestic 

American 

Germane 

Home 

Native 

Lidigenous  ' 

Pertinent 

Local 
Natural 

4 

Create 

Annihilate 

Abolish 

Demolish 

Disintegrate 

Destroy 

Dismember 

Exterminate 

Obliterate 

Undo 

Waste 

4 

Simple 

Complex 

Bright 

Complicated 

Confusing 

Compound 

Difficult 

Elaborate 

Gorgeous 

Intricate 

Grand 

Ornate 

Hard 

Wise 

Learned 
Sensible 
Smart 

4 

Extravagant 

Economical 

Careful 

Frugal 

Close 

Miserly 

Conservative 

Niggardly 

Economizing 

Parsimonious 

Penurious 

/ 

Saving 

Stingy 
Thrifty 

4 

Aristocratic 

Bourgeois 

Humble 

Common 

Low 

Democratic 

Lowly 

Plebeian 

Ordinary 

4 

Rare 

Abundant 

Commonplace 

Common 

Often 

Dense 

Ordinary 

Frequent 

Profuse 

Numerous 

Regular 

Plentiful 

Occasional 

Plenteous 

Usual 

4 

Dangerous 

Harmless 
Safe 

Peaceful 
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TABLE  X— Continued 


Value 


Stimulus  Word 


Correct  Responses        Half  Correct  Responses 


Slovenly- 

Neat 

Careful 

Neatly 

Carefully 

Tidy 

Gracefully 

Tidily 

Precise 
Trim 

Defective 

Complete 

Correct 

Faultless 

Effective 

Normal 

Good 

Perfect 

Healthy 

Sound 

Whole 

Stingy 

Generous 

Bounteous 

Lavish 

Bountiful 

Liberal 

Extravagant 

Prodigal 

Freehearted 

Magnanimous 

Unselfish 

Unsharing 

Wasteful 

Reveal 

Conceal 

Cover 

Hide 

Secrete 

Obscure 

Suppress 
Withhold 

Diligent 

Dilatory 

Careless 

Indolent 

Negligent 

Lazy 

Inattentive 

Slothful 

Idle 
Shiftless 

Join 

Abandon 

Detach 

Disjoin 

Disassociate 

Disconnect 

Disband 

Dismember 

Leave 

Divide 

Resign 

Part 

Tear 

Separate 

Undo 

Sever 

Untie 

Sunder 

Withdraw 

Impoverish 

Enrich 

Aggrandize 

Replenish 

Fertilize 

Nourish 
Strengthen 

Permanent 

Ephemeral 

Ephemerate 

Evanescent 

Fluctuating 

Fleeting 

Impermanent 

Passing 

Perishable 

Temporary 

Shifting 

Transient 

Transitional 

Transitory 

Unsubstantial 
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TABLE  X— Continued 


Value 

Stimulus  Word 

Correct  Responses 

Half  Correct  Responses 

7 

Elation 

Dejection 

Grief 

Depression 

Sadness 

Despair 

Shame 

Despondence 

Sorrow 

Despondency 

7 

Sinful 

Blameless 

Godly- 

Righteous 

Holy 

Sinless 

Innocent 

Just 

Religious 

Perfect 

Pious 

Pure 

Upright 

Virtuous 

7 

Obnoxious 

Agreeable 

Acceptable 

Beneficial 

Attractive 

Inoffensive 

Congenial 

Pleasing 

Desirable 

Harmless 

Helpful 

Likable 

Pleasant 

Winning 

7 

Conservative 

Extreme 

Aggressive 

Liberal 

Extravagant 

Progressive 

Extremist 

Radical 

Wasteful 

7 

Victorious 

Conquered 

Beaten 

Defeated 

Unsuccessful 

Vanquished 

Unvictorious 
Whipped 

7 

Obscure 

Clear 

Apparent 

Conspicuous 

Bright 

Eminent 

Disclose 

Lucid 

Distinct 

Plain 

Evident 

Prominent 

Famous 

Renowned 

Noticeable 

Reveal 

Notorious 

Obvious 

Unambiguous 

8 

Proficient 

Deficient 

Backward 

Inapt 

Incapable 

Unskilled 

Incompetent 

Unskillful 

Inefficient 

Lacking 
Unprepared 
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TABLE  X— Continued 


Value 


Stimulus  Word 


Correct  Responses        Half  Correct  Responses 


10 


ID 


II 


II 


II 


12 


Rigid 


Repulsion 


Imaginary 


Elastic 

Flexible 

Lax 

Lenient 

Limber 

Limp 

Plastic 

Relaxed 

Supple 

Yielding 

Attraction 
Cohesion 


Real 


Permit 

Forbid 
Prevent 
Refuse 
Restrain 

Orthodoxy 

Heresy 
Heterodoxy 

Analytical 

Synthetic 
Synthetical 

Extrinsic 

Intrinsic 

Sacred 

Defiled 
Profane 
Secular 
Unholy 

Dynamic 

Inert 
Static 

Loquacious 

Laconic 
Reticent 
Taciturn 
Silent 

Heterogeneous 

Homogeneous 

Easy 

Flabby 

Flaccid 

Loose 

Pliable 


Acceptance 
Agreeableness 
Admiration 
Love 

Actual 
Prosaic 
Realistic 
True 

Deny 

Disallow 

Disapprove 

Object 

Prohibit 

Catholicity 
Liberalism 
Unconventionality 
Unsoundness 

Unanalytical 


Essential 
Internal 

Common 

Desecrated 

Sacrilegious 

Unconsecrated 

Ungodly 

Vulgar 

Potential 

Powerless 

Weak 

Mute 
Reserved 
Quiet 
Untalkative 

Alike 
Similar 
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TABLE  X— Continued 


Value 

Stimulus  Word 

Correct  Responses 

Half  Correct  Responses 

12 

Spurious 

Authentic 

Actual 

Genuine 

Legitimate 

Real 

Pure 

Sound 

True 

Truthful 

Unadulterated 

13 

Disastrous 

Advantageous 

Favorable 

Beneficial 

Fortunate 

Helpful 

Gainful 

Safe 

Harmless 

Lucky 

Prosperous 

14 

Facility- 

Difficulty 

Awkwardness 

Clumsiness 

Disability 

Effort 

Inconvenience 

14 

Pride 

Humility 

Debasement 

Lowliness 

Degradation 

Meekness 

Humbleness 
Humiliation 

15 

Result 

Causation 

Beginning 

Cause 

Commencement 
Origin 
Purpose 
Start 

15 

Adroit 

Awkward 

Crude 

Clumsy 

Unskilled 

Inexpert 

Maladroit 

Unskillful 

17 

Parsimony 

Extravagance 

Freeheartedness 

Generosity 

Magnanimity 

Lavishness 

Wastefulness 

Liberality 

Prodigality 

22 

Suave 

Brusque 

Abrupt 

Impolite 

Blunt 

Tactless 

Crude 

Unpolished 

Gruff 

Harsh 

Rough 

Rude 

Uncouth 

Undiplomatic 

Unpleasant 

Untactful 

26 


Exoteric 


Esoteric 


Private 


VI. 

The  Computation  of  the  Relative  Difficulty 
OF  THE  Stimuli 

As  stated  previously,  when  the  test  was  administered,  each 
response  given  by  the  subject  as  well  as  the  reaction  time  of 
that  response  was  recorded.  On  the  basis  of  these  records  the 
following  data  were  tabulated  for  each  stimulus :  the  different 
responses,  the  reaction  time  of  each  response,  and  the  failures 
to  respond.  Records  were  so  kept  that  information  could  be 
obtained  not  only  regarding  the  number  of  subjects  giving  a 
certain  response  but  also  regarding  the  reaction  times  of  that 
response.  For  instance,  instead  of  merely  indicating  the  number 
of  people  who  gave  "young"  as  an  opposite  to  "ancient,"  the 
records  show  that  "yourig"  was  given  one  time  in  six-fifths  of 
a  second,  another  time  in  nine-fifths,  another  in  seven,  etc. 

Furthermore,  in  order  to  ascertain  when  a  sufficiently  large 
number  of  subjects  had  been  tested  for  the  results  to  be  reliable 
and  to  be  able  to  indicate  the  extent  of  this  reliability,  as  deter- 
mined by  the  change  brought  about  by  additional  sampling,  sepa- 
rate records  were  kept  of  all  tests  made  prior  to  the  fall  of  19 18. 
These  w^ill  be  designated  as  Group  I.  Group  II  consists  of  the 
fifty-six  records  secured  in  the  early  fall  of  19 18,  and  Group 
III  of  the  remaining  fifty-six  records.  Since  some  of  the  stimuli 
were  added  to  the  list  at  different  times  the  number  of  records  of 
subjects  in  Group  I  varies  from  zero  to  seventy -nine. 

After  the  acceptable  responses  had  been  determined  these 
record  sheets  w^ere  scored  and  the  number  of  correct  responses 
the  number  of  half  correct  responses,  and  the  number  of  failures, 
as  well  as  the  total  of  all  three,  were  tabulated.  The  number  of 
failures  includes  both  the  failures  to  give  any  opposite  and  the 
failures  to  give  the  correct  opposite.  These  results  are  to  be 
found  in  Table  XI,  columns  III,  IV,  V  and  VI.  Referring  to 
this  table,  we  read  that  for  the  word  "full,"  there  were  in  Group 


34  MARIE  HACKL  MEANS 

I,  sixty  correct  responses,  no  half  correct  responses,  and  three 
failures,  making  a  total  of  sixty-three  records  for  that  word  ob- 
tained before  the  fall  of  1918.  Reading  further  in  Group  II  for 
the  same  word,  we  have  fifty- four  correct  and  one  half  correct 
responses  with  one  failure,  thus  totaling  fifty-six.  In  Group  III 
we  find  the  following  distribution :  fifty-five  correct  and  one  half- 
correct. 

In  computing  the  percentage  of  failures,  to  be  found  in  column 
VII,  two  half-correct  responses  were  considered  equivalent  to  one 
failure.  The  formula  will  make  clear  the  procedure  followed. 

H 

F  -f  T 
Percentage  of  failures  =  ■ in  which 

F  :^  the  number  of  failures, 
H  =  the  number  of  half-correct  responses, 
N  =  the  total  number  tested. 
Applying  this  formula  to  the  word  "full"  we  have  in  Group  I, 

o 

3  +  - 
2 

^ =  .047 

63 
Group  II  does  not  represent  the  percentage  of  failures  in  Group 

II  alone  but  a  combination  of  Groups  I  and  II.     Applying  the 

4  +  -5 

formula  again  for  the  same  word  we  have    =  .037 

119 

in  which  4  equals  the  sum  of  the  failures  in  groups  I  and  II, 

.5  equals  half  of  the  half-correct  responses  in  groups  I  and  II, 

and  119  is  the  sum  of  the  total  number  tested.     Similarly  Group 

III  represents  the  sum  of  groups  I,  II,  and  III. 

This  procedure  of  estimating  the  three  different  percentages 
of  failures,  instead  of  only  one  was  adopted  because  it  indicated 
the  degree  of  reliability  of  the  measure.  Very  radical  changes 
obviously  would  mean  little  reliability.  Efforts  were  made  to 
have  our  sampling  as  representative  of  all  the  students  as  possible. 
Therefore  the  number  tested  include  graduate  students  as  well  as 
members  of  the  four  different  classes  of  undergraduates. 
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TABLE  XI 

Data   from  Which   the    Percentage   of   Failures    for  Each 
Stimulus  was  Computed 


I  II  III  IV  V 

Number  of  Number  of     Number 
Stimulus  Group      Correct    Half-Correct      of 

Responses    Responses      Failures 


VI  VII 

Total  Percentage 
Number  of 

Tested  Failures 


Full 


Negative 


After 


Dim 


Blunt 


Success 


Pessimistic 


Joy 


Public 


Profit 


Spend 


Always 


I 

II 
III 

I 

II 

III 

I 

II 

III 

I 

II 
III 

I 

II 
III 

I 

II 
III 

I 

II 

III 

I 

II 

III 

I 

II 
III 

I 

II 

III 

I 

II 

III 

I 

II 

III 


60 

54 
55 

48 

53 
56 

71 

55 
53 

66 
50 

42 

60 
48 
50 

59 
51 
44 

47 
49 
47 

65 

49 
48 

63 
46 
49 

55 
52 
41 

46 
SI 
51 

52 
44 
43 


o 
I 
I 

o 
o 
o 

o 

0 

o 

8 

3 
14 

o 

2 
O 

O 
I 

3 

o 
I 
o 

o 

3 
2 

o 
3 

3 

7 
0 

5 

9 
I 
I 

I 
O 
2 


3 
I 
o 

2 

3 
o 

3 
I 

3 

I 

3 
o 

3 
6 
6 

4 
4 
9 

2 
6 
9 

3 
4 
6 

5 
7 
4 

5 

4 

10 

8 
4 
4 

10 
12 
II 


63 

56 
56 

50 
56 
56 

74 
56 
56 

75 
56 

56 

63 
56 
56 

63 
56 
56 

49 

56 
56 

68 
56 
56 

68 
56 
56 

67 

56 
56 

63 
56 
56 

63 
56 
56 


.047 
•037 
.028 

.040 

.047 
.030 

.040 
.030 
.037 

.066 
.072 
.089 

.947 
.084 
.091 

.063 
.071 
.108 

.040 
.080 
.108 

•044 
.068 

.086 

•073 
.108 
.105 

.126 
.101 
•139 

.198 
.142 
.122 

.166 
.188 
.197 
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TABLE  XI— Continued 


I 

II 

III 

IV 

V 

VI 

VII 

Number  of 

Number  of 

Number 

Total 

Percentage 

Stimulus 

Group 

Correct 

Half-Correct 

of 

Number 

of 

Responses 

Responses 

Failures 

Tested 

Failures 

Graceful 

I 

50 

13 

5 

68 

.169 

II 

44 

7 

5 

56 

.161 

III 

45 

4 

7 

56 

.161 

Strength 

I 

56 

0 

8 

64 

■125 

II 

43 

0 

13 

56 

•  175 

III 

41 

0 

15 

56 

.204 

Ancient 

I 

22, 

26 

3 

62 

.258 

II 

32 

22 

2 

56 

•245 

III 

40 

14 

2 

56 

.218 

Expand 

I 

45 

I 

II 

57 

.201 

II 

43 

2 

II 

56 

.207 

III 

41 

0 

15 

56 

.227 

Barbarous 

I 

57 

II 

0 

68 

.080 

II 

34 

17 

5 

56 

■153 

III 

34 

12 

10 

56 

.194 

Hinder 

I 

52 

3 

13 

68 

.213 

II 

44 

0 

12 

s(> 

.213 

III 

39 

5 

12 

56 

.227 

Despondent 

I 

48 

12 

3 

63 

.142 

II 

2,7 

15 

4 

56 

.172 

III 

39 

8 

9 

56 

.191 

Vague 

I 

55 

2 

10 

67 

.164 

II 

42 

2 

12 

56 

.195 

III 

38 

I 

17 

56 

.231 

Fertile 

I 

54 

I 

IS 

70 

.221 

II 

40 

0 

16 

56 

.250 

III 

47 

0 

9 

56 

.222 

Doubtful 

I 

38 

9 

10 

57 

■254 

II 

39 

9 

8 

56 

.238 

III 

38 

10 

8 

56 

■226 

Injurious 

I 

48 

10 

10 

68 

.220 

II 

39 

8 

9 

56 

.225 

III 

36 

6 

14 

56 

•249 

Busy 

I 

39 

4 

24 

67 

■2^ 

II 

35 

I 

20 

56 

.378 

III 

41 

I 

14 

56 

•340 

Abstract 

I 

0 

0 

0 

0 

.000 

II 

35 

2 

19 

56 

.357 

III 

22 

4 

19 

56 

.365 
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TABLE  XI— Continued 


I 

II 

III 

IV 

V 

VI 

VII 

Number  of 

Number  of 

Number 

Total 

Percentage 

Stimulus 

Group 

Correct 

Half-Correct 

of 

Number 

.of        • 

Responses 

Responses 

Failures 

Tested 

Failures 

Advance 

I 

48 

0 

10 

58 

.172 

II 

32 

4 

20 

56 

^80 

III 

34 

I 

21 

56 

•314 

Foreign 

I 

27 

10 

6 

53 

.207 

II 

34 

ID 

12 

56 

.256 

III 

34 

10 

12 

56 

.272 

Create 

I 

50 

3 

14 

67 

.231 

II 

38 

I 

17 

56 

.292 

III 

35 

2 

19 

56 

.2g6 

Simple 

I 

13 

21 

9 

43 

453 

II 

34 

15 

7 

56 

.343 

III 

32 

14 

ID 

56 

.328 

Extravagant 

I 

41 

8 

7 

56 

.196 

II 

42 

6 

8 

56 

.196 

III 

35 

9 

12 

56 

^29 

Aristocratic 

I 

52 

4 

12 

68 

^5 

II 

34 

5 

17 

56 

.270 

III 

22 

I 

23 

56 

.316 

Rare 

I 

27 

16 

8 

61 

.262 

II 

38 

7 

II 

56 

.260 

III 

38 

6 

12 

56 

S62 

Dangerous 

I 

35 

I 

15 

51 

.303 

II 

26 

0 

20 

56 

■331 

III 

35 

0 

21 

56 

.346 

Slovenly 

I 

44 

10 

14 

68 

.279 

II 

27 

II 

18 

56 

•342 

III 

28 

4 

24 

56 

.380 

Defective 

I 

47 

6 

15 

68 

^4 

II 

38 

6 

12 

56 

.264 

III 

27 

8 

21 

56 

322 

Stingy 

I 

43 

II 

14 

68 

.286 

II 

30 

II 

15 

56 

.322 

III 

18 

13 

25 

56 

'397 

Reveal 

I 

38 

4 

21 

63 

.365 

II 

31 

4 

20 

56 

.378 

III 

34 

4 

18 

56 

•371 

Diligent 

I 

43 

4 

20 

67 

.328 

II 

33 

15 

8 

56 

•304 

III 

26 

9 

21 

56 

•351 
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TABLE  XI— Continued 


I 

Stimulus 

II 
Group 

III 

Number  of 

Correct 
Responses 

IV 

Number  of 

Half-Correct 

Responses 

V 

Number 

of 
Failures 

VI 

Total 

Number 

Tested 

VII 

Percentage 

of 

Failures 

Join 

I 

II 
III 

2,2 
38 
30 

II 
0 

7 

II 
18 
19 

55 
56 
56 

.300 
.310 
.341 

Impoverish 

I 

II 
III 

38 
22 
27 

4 
3 
0 

27 
31 
29 

69 

56 
56 

.420 
.492 
.500 

Permanent 

I 

II 
III 

2,7 
24 
34 

5 
I 

I 

21 

31 
21 

63 
56 

56 

•373 

462 

.437 

Elation 

I 

II 
III 

33 
17 
18 

10 
3 
3 

19 
36 

35 

62 

56 

.387 
.521 
.563 

Sinful 

I 

II 
III 

20 
13 
17 

40 

25 

19 

8 
18 
20 

68 
56 

56 

.411 

.471 
488 

Obnoxious 

I 

II 

III 

43 
14 
16 

II 
22 

14 

13 
20 
26 

67 
56 
56 

.276 
.402 
460 

Conservative 

I 

II 
III 

39 
15 
14 

3 
15 
IS 

27 
26 
27 

69 
56 
56 

•413 
496 
•533 

Victorious 

I 

II 
III 

43 
22 
24 

4 
9 
8 

24 
25 
24 

67 
56 
56 

•388 

•451 
466 

Obscure 

I 

II 
III 

27 

25 

25 

4 
3 
6 

26 

28 

25 

57 
56 
56 

491 

.508 

•505 

Proficient 

I 

II 
III 

18 
16 
14 

30 

18 

13 

15 
22 

29 

63 

S6 
56 

.476 
.512 
•551 

Rigid 

I 

II 
III 

27 
17 
14 

14 
18 
18 

16 

21 

24 

57 
56 

•403 
469 
.508 

Repulsion 

I 

II 
III 

29 
20 
18 

4 
2 

3 

34 
34 
35 

67 
56 
56 

•537 

•577 
.600 

Imaginary 

I 

II 
III 

22 
21 
20 

4 
2 
2 

28 
33 
34 

54 
56 

■555 
.581 
•596 
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TABLE  XI— Continued 


I                     II           III                IV                 V  VI                VII 

Number  of  Number  of     Number  Total  Percentage 

Stimulus           Group      Correct    Half-Correct      of  Number             of 

Responses    Responses      Failures  Tested  Failures 


Permit 

I 

II 

III 

28 
24 
24 

Orthodoxy 

I 

II 
III 

15 
9 
6 

Analytical 

I 

II 
III 

29 
10 
II 

Extrinsic 

I 

II 
III 

I 

16 
15 

Sacred 

I 

II 
III 

24 
15 
10 

Dynamic 

I 

II 

III 

5 
3 
7 

Loquacious 

I 

II 
III 

14 
13 
13 

Heterogeneous 

I 

II 
III 

2 

9 
12 

Spurious 

I 

II 
III 

6 

4 
4 

Disastrous 

I 

II 
HI 

18 
21 

Facility 

I 

II 

III 

10 

6 

5 

Pride 

I 

II 
III 

17 
8 
6 

Result 

I 

II 

III 

15 
10 

4 

12 

7 
8 


o 
o 
o 

I 
I 
o 

o 
o 
o 

4 

7 

II 

4 
4 
2 

20 
3 
5 

o 

4 
o 

12 
3 
3 

2 

6 
2 

I 

3 
o 

3 
2 
I 

15 
6 
6 


28 

25 
24 

40 
47 
50 

27 

45 
45 

I 

40 

41 

30 

34 
35 

30 
49 
47 

28 
40 
38 

I 
43 
44 

31 

49 
49 

30 

35 
33 

34 
47 
51 

53 
46 
49 

32 
40 
46 


68 
56 
56 

55 
56 
56 

57 
56 
56 


56 
56 

58 
56 
56 

39 
56 
56 

62 
56 
56 

3 

56 
56 

49 
56 
56 

SO 

56 
56 

45 
56 
56 

73 
56 
56 

62 
56 
56 


.500 

•504 
.502 

■727 
.783 
.820 

.482 
.646 
.698 

.500 
.706 
.719 

•551 
.609 

■647 

.820 

■873 
.867 

.612 

•673 
.689 

■333 
•779 
.782 

•755 
•833 
.857 

.620 
•650 
•635 

.766 
.821 
.853 

.746 
.786 
.816 

.636 
.698 
•755 


40 


MARIE  HACKL  MEANS 


TABLE  XI— Continued 


I  II  III  IV  V 

Number  of  Number  of     Number 
Stimulus  Group      Correct    Half-Correct      of 

Responses    Responses      Failures 


VI  VII 

Total  Percentage 

Number  of 

Tested  Failures 


Adroit 

19 

2 

40 

61 

.672 

II 

7 

3 

46 

56 

.756 

III 

3 

0 

53 

56 

.817 

Parsimony 

II 

2 

35 

48 

.750 

II 

7 

0 

49 

56 

.816 

III 

6 

6 

44 

56 

.823 

Suave 

2 

10 

34 

46 

.847 

II 

2 

9 

45 

56 

.867 

III 

I 

8 

47 

56 

.882 

Exoteric 

3 

0 

49 

52 

.942 

II 

I 

0 

55 

56 

.962 

III 

0 

0 

56 

56 

.975 

Since  the  value  of  the  stimuH  was  to  be  in  terms  of  both  ac- 
curacy and  speed,  the  median  time  of  all  responses,  both  correct 
and  half  correct,  was  computed.  The  time  score  taken  with  a 
stop-watch  represents  fifths  of  a  second.  As  in  the  case  of  the 
accuracy  score,  three  time  scores  were  reckoned  for  each  word, 
the  last  score  representing  the  combined  time  of  all  three  groups. 
See  column  III,  Table  XII.  The  percent  of  failures  of  the  pre- 
ceding table  is  copied  in  order  to  make  clear  the  process  by  which 
the  values  of  the  several  stimuli  are  reached. 

To  arrive  at  the  relative  difficulty  of  the  stimuli,  the  percent- 
tage  of  failures  was  multiplied  by  the  median  time.  If  credit  is 
to  be  given  for  both  measures,  some  more  or  less  arbitrary 
scheme  must  be  adopted.  This  procedure  seemed  justifiable  since 
by  it  words  of  equal  accuracy  scores  received  different  point 
values  if  the  difference  in  time  varied  to  any  appreciable  extent. 
This  was  precisely  what  was  desired.  Any  method  which  covered 
up  this  difference  was  to  be  avoided. 

In  order  to  determine  a  point  value  for  the  stimuli,  the  stan- 
dard deviation  of  the  different  values,  as  indicated  in  the  fourth 
column  of  the  following  table,  was  obtained.  Beginning  at  the 
average  and  calling  it  zero,  steps  of  one-fifth  standard  deviation 
were  then  made,  positively  and  negatively.     These  steps  being 
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equal  in  a  sense,  the  f  urtherest  negative  step  was  assigned  a  value 
of  one,  the  next  two,  and  so  on  up  to  twenty-six,  the  most  difficult 
word  in  the  list.  These  point  values  are  given  in  the  fifth  column 
of  Table  XII. 

In  selecting  the  stimuli  for  the  completed  test,  those  words 
v/hich  had  received  a  hundred  percent  accuracy  score  were  dis- 
carded, for  thus  we  were  better  enabled  to  locate  a  zero  point. 
Obviously  we  cannot  find  or  infer  the  place  where  the  ability 
to  give  opposites  begins,  as  we  can  in  the  physical  sciences  for 
weight  or  length.  Therefore  the  zero  point  will  of  necessity  be 
an  arbitrary  one  selected  to  represent  very  little  of  the  trait  in 
question.  In  eliminating  all  words  with  a  hundred  percent  ac- 
curacy score,  but  retaining  those  next  in  difficulty,  we  assume 
that  a  college  student  who  is  unable  to  score  a  point  on  the  test 
as  it  now  stands  has  zero  ability  in  naming  opposites.  Since  we 
have  located  the  distance  from  the  median  in  terms  of  standard 
deviation,  we  have  equally  difficult  steps.  Roughly  we  assigned 
to  the  furtherest  negative  step  a  value  of  one.  Since  we  are  not 
concerned  as  were  Woody^  and  Hering^  with  finding  the  dis- 
tances between  different  grades  or  groups  of  pupils,  it  seems 
impracticable  to  be  too  particular  about  the  absolute  point  of 
the  zero.  In  addition  any  error  in  slightly  misplacing  the  point 
might  be  of  less  importance  than  that  made  in  weighting  time 
and  accuracy. 

It  is  regretted  that,  especially  in  the  case  of  the  more  difficult 

words,  the  measure  of  time  is  based  on  so  small  a  number  of 

records.  As  an  indication  of  the  reliability  of  the  median  time,  we 

have  computed  the  median  deviation  and  coefficient  of  variability 

for  each  word.     The  probable  error  of  the  median  is  calculated 

5  Q  dis 

by  the  formula,  P.  E.  med.  = ^ 

4\An 

1  Woody,  Clifford.  "Aleasurements  of  Some  Achievements  in  Arithmetic," 
Col.  Univ.  Cont.  Educ.  1916,  LXXX. 

2  Hering,  John  P.,  "Derivation  of  a  Scale  to  Measure  AbiUties  in  Scientific 
Thinking,"  Jour.  Educ.  Psychol.,  1919,  IX,  417-431. 

3  Thorndike,  E.  L.,  "Mental  and  Social  Measurements,"  New  York,   1916, 
P-  195- 
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TABLE  XII 
The  Point  Value  of  the  Stimuli  Computed  from  Table  IX 

I  n         in        IV        V        VI       vn       viii 

Percent    Median  Relative     Point     Median  Coefficient  Probable 

Stimulus  of  Time      Value     Value  Deviation        of        Error  of 

Failures  Variability  Median 


1.48  .21  .14 


Full 

.047 

.037 
.028 

7 
6 

7 

.329 
.222 
.209 

Negative 

.040 

.047 
.030 

7 
7 
7 

.280 

•329 
.210 

After 

.040 
.030 
.037 

6 
7 

7 

.240 
.210 
.259 

Dim 

.066 
.072 
.089 

8 

7 
7 

.528 
•504 
.623 

Blunt 

.047 
.084 
.091 

6 

7 
7 

.282 
.588 
.^Z7 

Success 

.063 
.071 
.108 

6 

7 

6 

.378 
■497 
.648 

Pessimistic 

.040 
.080 
.108 

6 
6 
6 

.240 

.480 
.648 

Joy 

.044 
.068 
.086 

7 
8 
8 

.308 

•544 
.688 

Public 

•073 
.108 

.105 

7 
7 
8 

•511 
.756 
.840 

Profit 

.126 
.101 
•139 

7 
8 
8 

.882 

.808 

1.112 

Spend 

.198 
.142 
.122 

10 
10 
10 

1.980 
1.420 
1.220 

Always 

.166 
.188 
.197 

8 
8 
8 

1.328 
1.504 
1.576 

Graceful 

.169 
.161 
.161 

10 
10 
10 

1.690 
1.610 
1. 610 

1.22  .17  .12 


1.20  .17  .II 


1.64  .23  .15 


1.40  .20  .13 


1.50  .25  .14 


I. II  .18  .II 


2.08  .26  .20 


2.03  .25  .19 


2.26  .28  .22 


2,80  .28  .27 


1-90  .23  .19 


Z-2^  -ZZ  .32 
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TABLE  XII— Continued 


II  III           IV            Y           VI           VII          VIII 

Percent  Median  Relative     Point     Median  Coefficient  Probable 

of  Time      Value     Value  Deviation        of        Error  of 

Failures  Variability  Median 


Stimulus 


Strength 

.125 
.175 
.204 

7 
8 
8 

.875 
1.400 
1.632 

Ancient 

.258 

.245 
.218 

8 

7.5 
8 

2.064 
1.837 
1-744 

Expand 

.201 

.297 
.227 

9 
8 
8 

1.809 
1.656 
1.816 

Barbarous 

.080 
.153 
.194 

II 
II 
II 

.880 
1.683 
2.134 

Hinder 

.213 

.213 
.227 

9 
10 
10 

I.917 
2.130 
2.270 

Despondent 

.142 
.172 
.191 

14 
12 
12 

1.988 
2.064 
2.292 

Vague 

.164 
.195 
.231 

9 
10 
II 

1.476 
1.950 

2.541 

Fertile 

.221 
.250 
.222 

12 
12 
12 

2.652 
3.000 

2.664 

Doubtful 

.254 
.238 
.236 

12 
12 
12 

3-048 
2.858 
2.832 

Injurious 

.220 
.225 
.249 

II 
12 
12 

2.420 
2.700 
2.988 

Busy 

.388 
.378 
.340 

10 

10 
9 

3.880 
3.780 
3.060 

Abstract 

.000 

.357 
.365 

0 
9 

8.5 

0.000 

3.213 
3-102 

Advance 

.172 
.280 
■314 

9 

9 

10 

1.548 

2.520 
3.140 

2  1.95  .24  .20 


3  2.30  .25  .22 


3  1.28  .16  .13 


3  2.90  .26  .28 


3  348  .34  .36 


3  3.54  -29  -35 


3  2>-22>  .29  .34 


3  3.85  .32  .40 


4  3.89  .32  .40 


4  4-47  .37  .46 


4  2.64  .29  .30 


4  2.10  .24  .30 


4         2.90         .29         .z:^ 
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TABLE  XII— Continued 


Stimulus 


II  III  IV  V  VI  VII  VIII 

Percent    Median  Relative     Point     Median  Coefficient  Probable 

of           Time      Value     Value  Deviation        of        Error  of 
Failures Variability  Median 


Foreign 

.207 
.256 
.272 

12 
12 
12 

2.484 
3.072 
3.264 

Create 

.231 
.292 
.296 

10 

II 
II 

2.310 

3-212 
3-256 

Simple 

•453 
•343 
.328 

8 
10 
10 

3.624 
3-430 

3.280 

Extravagant 

.196 
.196 
.229 

14 
15 
15 

2.744 
2.940 

3-435 

Aristocratic 

•20s 
.270 
.316 

12 
II 
II 

2.460 
2.970 
3-476 

Rare 

.262 
.260 

.262 

14 
14 
14 

3-668 
3-640 
3-668 

Dangerous 

•303 
•331 
.346 

II 
II 
II 

3-333 
3-641 
3.806 

Slovenly 

•279 
■342 

.380 

II 
12 
II 

3.069 
4.104 
4.180 

Defective 

.264 
.264 
.322 

13 
13 
13 

3-432 
3-432 
4.186 

Stingy 

.286 
■322 
•397 

10 
12 
II 

2.860 
3-864 
4-367 

Reveal 

.365 
.378 
-371 

II 
II 

12 

4-015 
4.158 

4-452 

Diligent 

.328 
•304 
•351 

13 
13 
13 

4.264 
3-952 

4-563 

Join 

.300 
•310 
•341 

14 
13 
14 

4.200 
4.030 
4-774 

3.80 


4  3-31 


5  3-62 


•31 


5  3-83  -St 


•45 


-30  .36 


4  3.23  .32  .28 


4  4-14  -27  .43 


4  4-20  .38  .46 


4  3-39  -24  -35 


4  3-75  -34  -45 


-32  .63 


5  5.29  -40  .58 


5  3-34  -30  .37 


-44 


5  3-97  .30  .43 


5  4.90  .35  .56 
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TABLE  XII— Continued 


I                    II  III            IV 

Percent  Median  Relative 

Stimulus                  of  Time      Value 
Failures 


V  VI  VII  VIII 
Point     Median  Coefficient  Probable 
Value  Deviation        of        Error  of 
Variability  Median 


Impoverish 

.420 
•492 
.500 

12 
II 
II 

5.040 
5-412 
5-500 

Permanent 

■272 
,462 
.437 

13 

13 
14 

4.849 
6.006 
6.118 

Elation 

.387 
.521 
•563 

10 
12 
12 

3-870 
6.262 
6.756 

Sinful 

.411 
.471 
.488 

13 
16 

14 

5-343 
7.536 
6.832 

Obnoxious 

.276 
.402 
.460 

13 

14 
15 

3.588 

5-628 
6.900 

Conservative 

.413 
.496 

.533 

II 
II 
13 

4.543 
6.448 
6.929 

Victorious 

.388 

.451 
.466 

14 
15 
15 

5-432 
6.765 
6.990 

Obscure 

.491 
.508 

.505 

12 
13 
14 

5.892 
6.604 
7.070 

Proficient 

-476 
.512 
■551 

II 
13 
13 

5.236 
6.656 
7.163 

Rigid 

■403 
469 
.508 

15 
IS 

15 

6.045 
7.035 
7.620 

Repulsion 

•537 
•577 
.600 

II 

13 
13 

5-907 
7-501 
7.800 

Imaginary 

•555 
.581 
.596 

II 
12 
14 

6.105 
6.972 
8-344 

Permit 

.500 

.504 
.502 

16 
16 
17 

8.000 
8.064 
8.534 

6  .341  .31 


•49 


7  4.50  -32  .55 


7  3-79  .31  .51 


7  5-10  .36  .64 


7  5.36  .35  .61 


7  4.30  .33  -53 


7  4-75  .31  .56 


7  6.06  43  .79 


8  5-45  .41  .65 


8  4-97  .33  -59 


8  5-30  .40  .75 


9  4-56  .32  .67 


9  6.50  .38  .80 
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TABLE  XII— Continued 


Stimulus 


II 

Percent 

of 
Failures 


III  IV 

Median  Relative 

Time      Value 


V  VI  VII  VIII 
Point    Median  Coefficient  Probable 
Value  Deviation        of        Error  of 
Variability  Median 


Orthodoxy 

Analytical 

Extrinsic 

Sacred 

Dynamic 

Loquacious 

Heterogeneous 

Spurious 

Disastrous 

Facility 

Pride 

Result 

Adroit 


•727 
.783 
.820 

.482 
.646 
.698 

.500 
.706 
.719 

•551 
.609 

.647 

.820 

.873 
.867 

.612 

.673 
.689 

•333 
•779 
.782 

.755 
•833 
•857 

.620 

•650 
■635 

.766 
.821 
.853 

•746 

.786 
.816 

.636 
.699 
•755 

.672 
•756 
.817 


II 
II 
II 

16.5 

14 

14 

9 
17 
14 

15 
16 
16 

12 
12 
12 

15 
15 
16 

145 

13 

15 

14 
14 
14 

17 
19 
20 

16 
18 
16 

16 
17 
17 


7-997 
8.613 
9.020 

7-953 
9.044 
9.772 

4500 
12.000 
10.066 

8.265 

9-744 

10.352 

9.840 
10.476 
10.404 

9.180 
10.095 
11.024 

4.828 
10.127 
11.730 

10.570 
11.662 
11.998 

10.540 
12.350 
12.700 

12.256 
14.778 
13-648 

1 1^936 

I3^362 
13^872 


20  12.740 

22  15.378 

20  15.100 

16  10.752 

17  12.852 
18.S  15.114 


9     2.50     .22     .57 


10     4.27     .30     .74 


10     5.30     .37    1.17 


II     6.17     .38     .91 


II     3-75     .31     -93 


II     5.08     .31     .76 


12     4.88     .32    1. 17 


12     5.50     .39    1. 21 


13     708     .35    1. 10 


14     7-00     .43    1.75 


14     5-88     .34    1.20 


IS     8.00     .40    1.33 


15     6.23     .33    1.33 
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TABLE  XII— Continued 


I                  II  III           IV 

Percent  Median  Relative 

Stimulus                  of  Time      Value 
Failures 


V           VI  VII  VIII 
Point     Median  Coefficient  Probable 
Value  Deviation        of        Error  of 
Variability  Median 


Parsimony 


Suave 


Exoteric 


.750 
.816 
.823 

.847 
.867 
.882 

.942 
.962 
.975 


25 
22 
21 


36 

27-5 

27-5 


18.750 
17-952 
17.183 


24.5  20.751 
25  21.675 
25        22.050 


33.912 

26.455 
26.812 


6.60 


22  9.17 


26  9.50 


.31  1.45 


.36         2.02 


•34  5-93 


VII. 

The  Conversion  into  a  Group  Test 

In  order  to  make  it  possible  for  even  the  inexperienced  to  use 
the  test  with  faciHty,  it  was  converted  into  a  group  test.  Thus, 
obviously,  much  time  in  administering  and  labor  in  scoring  were 
dispensed  with.  In  addition,  the  personal  element  became  almost 
negligible.  The  words  were  arranged  in  the  ascending  order  of 
difficulty  and  six  minutes  allowed  for  the  test.  This  time  limit 
was  determined  empirically,  after  experimenting  upon  a  dozen 
or  more  students  at  the  George  Peabody  College  and  some  sixty 
students  at  the  North  Georgia  Agricultural  College.^  In  arriving  at 
the  time  to  be  allowed,  two  considerations  were  held  in  mind.  It 
was  desired  that  sufficient  time  be  given  for  individual  scores  to 
cover  a  considerable  range,  and  thus  avoid  a  large  undistributed 
group  of  poor  records.  On  the  other  hand  the  time  should  be  so 
planned  that  a  perfect  score  is  impossible.  Two  individuals  with 
a  perfect  score  might  vary  in  efficiency,  but  under  the  conditions 
of  the  test  no  measure  of  their  difference  could  be  obtained. 

The  test  sheets  were  printed  upon  a  good  quality  of  white 
paper,  eight  and  a  half  by  eleven  inches.  The  directions  printed 
on  the  back  of  the  sheet  explained  to  the  experimenter  as  well  as 
to  the  subject,  exactly  what  was  to  be  done.  A  copy  is  included 
in  the  Appendix.  By  mistake  the  word  "rare"  was  placed  be- 
tween "defective"  and  stingy."  It  should  follow  "aristocratic." 
In  scoring  the  papers,  however,  proper  credit  was  assigned  the 
word.  The  displacement  was  so  slight  that  it  is  highly  improb- 
able that  the  final  results  were  at  all  affected. 

Three  thousand  copies  of  the  test  were  sent  to  psychologists 
in  twenty  different  normal  schools,  colleges,  and  universities 
throughout  the  country.  Of  this  number,  one  thousand  six  hun- 
dred and  twenty-eight  copies  were  returned   for  grading  from 

1  These  results,  which  were  obtained  through  the  kindness  of  Professor 
George  Camp,  are  not  included  in  the  norms  because  the  sheets  were  mimeo- 
graphed instead  of  printed. 
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fourteen  widely  different  institutions.  Unfortunately,  a  number 
of  papers  arrived  too  late  to  be  included  in  the  norms  herein  pre- 
sented. The  table  below  indicates  the  number  of  subjects  tested 
in  each  school  and  the  class  to  which  both  the  men  and  women 
then  belonged. 

TABLE  XIII 

The  Number  of  Men  and  Women  Tested  in  Each  School  and  Class  in 

Which  They  Were  Enrolled 


Freshmen 

Soph's 

Juniors 

Seniors 

Graduates 

Total 

Women  Men 

Women  iMen 

Women  Men 

Women 

Men 

Women 

Men 

Wellesley 

College 

43 

95 

138 

Brooklyn 

Training  School 

148        5 

153 

University 

of  Illinois 

4        2 

44 

46 

60      34 

35 

16 

2 

243 

University 

of  Montana 

23 

2 

10        5 

24 

I 

66 

Alabama 

State  Normal 

50 

50 

University 

of  Chicago 

2          6 

40 

74 

7       13 

20 

9 

2 

173 

Teachers' 

College 

2 

13 

2 

9 

28 

54 

Vanderbilt 

University 

I 

I 

12 

14 

University 

of  Kansas 

I 

32 

27 

II        6 

3 

4 

I 

85 

University 

of  Tennessee 

I 

25 

19 

7        2 

4 

2 

60 

University 

of  Louisiana 

2           I 

14 

9 

5      14 

8 

4 

I 

58 

Fitchburg 

Normal  School 

54      I" 

26 

17        2 

116 

Kansas 

State  Normal 

49        6 

38 

I 

15        3 

112 

Ohio   State 

Universitv 

59      (>3 

40 

58 

33      22 

14 

13 

2 

2 

306 

Total 

319     lOI 

375 

237 

262     102 

121 

51 

16 

44 

1628 

The  final  results  of  the  sixteen  hundred  records  indicated  class 
variations  as  well  as  slight  sex  differences.  In  order  to  ascertain 
a  possible  cause,  the  writer  has  computed  the  percentage  of  stu- 
dents of  both  sexes  in  each  of  the  classes.  This  distribution  is 
shown  in  the  following  table. 

In  grading  the  papers  two  classes  of  errors  were  encountered, 
namely,  the  misspelled  word,  and  the  abbreviated  word.  In  the 
former  case  full  credit  was  allowed  whenever  the  spelling  clearly 


Women 

.291 

Men 

.188 

Both  Sexes 

■■27 
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TABLE  XIV 
The  Percentage  of  Students  of  Both  Sexes  in  Each  of  the  Classes 

Freshmen      Sophomores       Juniors  Seniors      Graduates 

.343  .239  .110  .014 

.440  .190  .095  -082 

.37  -22  .10  .03 

showed  that  an  acceptable  word  was  meant.  However,  when  a 
word  similar  in  sound  was  correctly  spelled  no  credit  was  given. 
As  an  example,  "save"  was  not  accepted  as  an  opposite  of  "dan- 
gerous," but  "richous"  (actual  case)  as  an  opposite  of  "sinful" 
was  given  full  credit.  This  rule  was  adopted  because  without  it 
gradings  by  different  individuals  would  vary  considerably. 

Where  the  response  was  abbreviated  it  was  given  no  credit. 
The  only  two  abbreviations  were  "opt"  as  the  opposite  of  pes- 
simistic and  "dem"  as  opposite  of  aristocratic.  In  both  cases 
the  subject  might  have  intended  to  write  "optimist"  and  "demo- 
crat," both  of  which  would  have  been  scored  a  failure.  Even 
had  he  had  in  mind  the  adjective  instead  of  the  noun,  his  score 
would  have  been  raised  only  five  points.  No  doubt  in  abbreviat- 
ing the  words,  he  was  thus  enabled,  at  an  unfair  advantage  to 
others,  to  write  the  opposite  of  a  more  difficult  word.  In  view  of 
the  fact  that  a  time  limit  was  placed  on  the  test,  not  to  penalize 
abbreviations  seems  unjust  to  those  students  who  have  followed 
the  directions  carefully.  Fortunately  the  number  of  students 
abbreviating  was  negligible. 

The  sum  of  the  point  values  of  the  stimuli  as  indicated  in  Table 
XII  is  four  hundred  and  thirty-eight.  Since  each  response  may 
receive  the  assigned  value,  one-half  that  value,  or  zero,  almost 
any  score  between  zero  and  this  number  is  possible.  This  wide 
range  of  distribution  eliminates  the  probability  of  a  large  group 
of  subjects  of  different  ability  receiving  identical  scores.  The 
actual  scores  based  on  the  records  obtained  varied  from  zero  to 
three  hundred  and  fifty-one. 


VIII. 

The  Establishment  of  Norms  for  College  Students 

Sixteen  hundred  and  twenty-eight  test  papers  were  graded  and 
the  norms  for  college  students  by  the  group  test  were  based  on 
these  records.  In  order  to  ascertain  the  form  of  the  distribution 
of  the  scores  made  by  the  several  classes,  steps  of  fifty  beginning 
with  zero  were  made  and  the  percent  of  students  included  between 
these  steps  was  estimated.  Examination  of  Table  XV  reveals 
the  fact  that  the  largest  percentage  of  scores  falls  between  one 
hundred  and  one  hundred  fifty.  This  is  perhaps  more  clearly 
shown  in  the  graphs  (Figures  2,  3,  4,  5,  6)  based  on  this  table. 

TABLE  XV 

Percent  of  Students  of  Several  Classes  Attaining  Scores 
Within  the   Indicated   Limits 


Score 

Freshmen 

Sophomores 

Juniors 

Seniors 

Graduates 

0-  49-5 

.069 

•033 

.016 

.005 

.000 

50-  99.5 

.356 

.272 

.190 

.184 

.066 

100-149.5 

•397 

.346 

.318 

.406 

.300 

150-199-5 

.124 

.225 

.247 

.214 

.250 

200-249.5 

•045 

.078 

.156 

.III 

.163 

250-299.5 

.007 

.032 

.063 

.047 

.133 

300-349.5 

.002 

.014 

.010 

.028 

.086 

350-399-5 

.000 

.000 

.000 

.005 

.000 

The  norms  for  college  students  are  presented  in  the  form  of 
percentile  tables  because  this  method,  statistically  simple,  admits 
of  convenient  use  and  permits  comparisons  to  be  made  with 
other  tests,  similarly  treated.  Let  us  suppose  that  a  directions 
test  has  been  standardized  and  the  norms  compiled  in  terms  of 
percentiles.  Scores  made  in  the  opposites  test  can  be  compared 
with  scores  made  in  the  directions  test  when  both  are  converted 
into  percentiles.  The  two  percentiles  can  be  averaged  or  the 
median  computed.  This  is  statistically  impossible  when  the 
two  scores  are  given  in  terms  of  the  score  in  points  made  on 
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Figure  6 

The  vertical  line  in  the  graph  shows  the  percentage  of  graduates 
receiving  the  scores  indicated  on  the  base  line. 

each  test.  Obviously,  if  the  possible  grades  on  the  opposites  test 
range  from  zero  to  four  hundred  and  thirty-eight  and  on  the 
hypothetical  directions  test  from  zero  to  sixteen,  to  add  or 
average  the  two  scores  made  by  any  individual  would  practically 
amount  to  weighting  the  opposites  test.  But  the  percentile 
scores  can  be  averaged  and  the  final  result  will  represent  a  com- 
bination of  the  two  tests  to  which  equal  credit  is  given. 

In  order  to  make  clearer  the  meaning  of  the  percentile  table 
the  following  explanation  of  the  manner  in  which  it  was  derived 
is  offered.  For  instance  in  the  case  of  the  freshmen,  as  indicated 
in  Table  XVI,  all  scores  were  arranged  in  numerical  order,  be- 
ginning with  the  lowest  and  ending  with  the  highest.  The  num- 
ber of  scores  was  ascertained,  which  in  this  case  was  four  hun- 
dred and  twenty.  Ten  percent  of  the  number  of  scores  was 
counted  off,  beginning  with  the  lowest  score.  The  score  upon 
which  the  forty  second  count  fell  became  the  tenth  percentile.    A 
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glance  at  the  table  will  show  that  in  this  instance  it  was  fifty- 
eight  and  a  half.  This  means  that  this  grade  is  not  exceeded 
by  the  lowest  ten  percent  of  the  freshmen,  and  that  any  student 
making  this  grade  is  equalled  or  surpassed  by  ninety  percent  of 
the  college  students. 

An  additional  ten  percent  is  counted  off  to  secure  the  next 
higher  percentile,  and  so  on.  • 

TABLE  XVI 

Norms  for  College  Students  in  Terms  of  Percentile  Scores  in  the 

Group  Test 


Percentile 

Freshmen 

Sophomores 

Juniors 

Seniors 

Graduates 

All  Comb'd 

10 

58-5 

66.5 

77-0 

84.0 

107.0 

68.5 

20 

74-5 

84-0 

97.0 

lOO.O 

124.0 

78.0 

30 

87.0 

99-0 

1 16.0 

II6.5 

138-5 

lOI.O 

40 

97-0 

1 12.0 

130-5 

125.5 

151.0 

1 14-5 

50 

1070 

125.0 

146.0 

137-5 

155-5 

12S.5 

60 

120.0 

139-9 

161.5 

147.0 

191-5 

142.0 

70 

132-5 

156.0 

181.0 

166.0 

200.0 

1^7.0 

80 

147.0 

17S-5 

203-0 

I87.S 

247.0 

181.5 

90 

172.0 

214.0 

239-0 

23 1 -0 

298.0 

220.5 

100 

303-5 

338.0 

3130 

35 1 -0 

313-0 

3510 

Number 

Tested 

420 

612 

364 

172 

60 

1628 

Frequently  comparisons  of  different  groups  are  stated  in  terms 
of  the  percent  of  individuals  of  the  first  group  reaching  or  ex- 
ceeding the  twenty-five  or  seventy-five  percentile  of  the  second 
group.  In  the  table  below  the  twenty-five,  fifty,  and  seventy-five 
percentiles  are  indicated  for  each  class. 

TABLE  XVII 
Twenty-five,  Fifty,  and  Seventy-five  Percentile  Scores  for  Each  Class 


Percentile 

Freshmen  Sophomores  Juniors 

Seniors 

Graduates 

All  Comb'd 

25 
50 
75 

82.5              92.5            106.0 
107.0            125.0            146.0 
139-0            165.5            192.0 

K)8.0 

137-5 
1730 

136.0 

155-5 
213-0 

89.5 
128.5 
169.0 

The  sophomores  surpass  the  freshmen  and  they  are  in  turn 
outstripped  by  the  juniors.  The  gap  between  seniors  and  gradu- 
ates is  wide;  but  the  former  are  little  if  at  all  superior  to  the 
juniors.  A  consideration  of  the  lowest  and  highest  score  made 
by  the  several  classes  reveals  the  fact  that  although  ten  percent  of 
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the  juniors  barely  reach  the  lowest  score  made  by  the  graduates, 
still  the  highest  score  attained  by  each  class  is  identical.  In 
Table  XVIII  is  shown  the  gradual  rise  of  the  lowest  score  real- 
ized by  the  different  classes,  contrasted  with  the  irregularity  of 
the  highest  score. 

TABLE  XVIII 
Table  Showing  the  Highest  and  Lowest  Score  Made  by  Each  Class 


Freshmen  Sophomores  Juniors 

Seniors    Graduates 

T^owest   Score    . . . . 

o                 i8.=;              "^S.o 

46.0              75.0 
3Sr-0            313-0 

Highest  Score    . . . 

303.5            338.0            313-0 

Although  men  are  slightly  superior  in  the  test,  to  women 
belongs  the  highest  as  well  as  the  lowest  score.  The  greatest 
sex  difference  is  to  be  found  in  the  fortieth  and  fiftieth  percentiles 
which  in  each  case  are  surpassed  by  the  next  lowest  percentile 
secured  by  the  men.  The  table  below  shows  the  ten  percentiles 
for  both  sexes  and  the  number  tested  in  each  case.  It  also  in- 
cludes, for  the  benefit  of  those  interested  in  sex  differences,  the 
more  commonly  employed  twenty-five  and  seventy-five  per- 
centiles. 

TABLE  XIX 
Percentile  Scores  for  Men  and  Women 

Percentiles Men Women 

o 

10  72.0  60.0 

20  92.5  85.5 
25  99.0  92.0 
30  106.0  98.5 
40  122.5  lor.o 
SO  133-5  II 50 
60  147.0  139.0 
70  163.S  155.5 
75  176.0  167.0 
80  189.0  178.0 
90  221.0  222.0 
100 338.0 351.0 

Number  Tested 535 1093 

A  consideration  of  the  range  of  scores  made  by  both  sexes 
would  seem  to  indicate  that  women  have  a  wider  range  than 
men  in  the  trait  in  question.  Whereas  men  range  only  from 
18.5  to  338,  women  beginning  with  an  initial  score  of  o  reach 
351  points  as  their  highest  score. 


IX. 

Interpretation  of  the  Results 

Whether  the  test  is,  as  Simpson^  beheves,  a  selective  thinking 
test,  we  cannot  from  the  very  nature  of  the  data  conclude.  How- 
ever, we  feel  justified  in  assuming  that  it  is  a  test  success  in 
which  is  dependent  upon  native  ability  rather  than  number  of 
years  schooling.  The  higher  scores  attained  by  the  graduates 
confirm  rather  than  refute  the  contention,  for  although  there  is 
a  constant  process  of  weeding  out  the  mentally  inferior,  selection 
operates  particularly  at  the  conclusion  of  work  offered  for  the 
bachelor's  degree.  The  graphs  (figures  2,  3,  4,  5,  and  6)  reveal 
a  considerable  amount  of  overlapping  from  year  to  year.  Be- 
tween ten  and  twenty  percent  of  the  freshmen  reach  the  median 
score  of  the  graduates  and  practically  thirty  percent,  the  median 
score  of  the  seniors. 

Not  without  significance  is  the  range  of  scores  of  the  several 
classes.  An  examination  of  Table  XVIII  discloses  the  fact  that 
the  highest  individual  score  obtained  by  either  a  junior  or  gradu- 
ate is  surpassed  by  a  sophomore.  This  condition  is  not  to  be  at- 
tributed to  the  simplicity  of  the  test,  which  might  admit  of  a 
number  of  nearly  perfect  records.  On  the  contrary  three  hundred 
and  fifty-one  is  the  highest  score  reached,  whereas  a  possible 
four  hundred  and  thirty-eight  could  be  attained  in  the  allotted 
time  were  an  individual  familiar  with  the  opposites.  The  lowest 
score  made  by  the  several  classes  ascends  regularly  from  fresh- 
men to  graduates,  all  of  which  seems  to  indicate  that  instead 
of  the  scores  progressing  in  absolute  value  as  the  years  in  school 
increase,  the  limit  of  attainment  is  about  reached  in  the  freshmen 
year,  and  thereafter  a  dropping  off  of  the  poorer  students  raises 
the  percentage  of  high  scores  made  by  the  more  advanced 
students. 

The  form  of  the  distribution  of  the  scores,  skewed  as  it  is  to 

^  Simpson,  Benjamin  R.,  Op.  cit. 
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the  lower  end,  indicates  that  the  time  allotted  is  too  short.  Had 
eight  minutes  instead  of  six  been  allowed  for  the  test,  the  scores 
would  have  been  scattered  over  a  wider  area  and  the  form  of 
distribution  would  probably  have  approached  the  normal.  As  it 
stands  the  test  provides  a  large  number  of  possible  scores  which 
can  be  attained  by  the  college  student.  In  addition,  because  of 
its  difficulty,  it  offers  an  opportunity  of  testing  a  more  highly 
selected  group. 

Sex  differences  favor  the  male.  King  and  M'Crory^  found 
that  the  same  condition  obtained  with  the  hard  opposites  test 
they  used.  In  the  present  investigation  twice  as  many  women 
were  tested  as  men.  An  equal  number  might  alter  the  situation. 
This  difference  might  be  due  to  the  fact  that  the  normal  schools 
largely  attended  by  women  were  slightly  inferior  in  the  test  to  the 
colleges  and  universities.  In  addition,  proportionally  more 
women  belonged  to  the  freshman  class  and  fewer  were  enrolled 
in  the  graduate  school,  as  is  indicated  in  Table  XIV.  Until  an 
equal  number  of  unselected  cases  from  each  sex  is  tested,  the 
subject  is  debatable. 

2  King,   Irving,   and  M'Crory,  J.   L.,   "Freshmen   Tests   at   the    State  Uni- 
versity of  Iowa,"  Jour.  Educ.  Psychol.,  1918,  IX. 


X. 

Suggestions  and  Directions  for  the  Use  of  the 

Opposites  Test 

The  following  suggestions  are  recommended  for  using  the 
test:  Distribute  the  test  sheets  with  the  face  downward  and 
read  aloud  the  written  directions  found  on  the  back  of  the  sheets. 
In  timing  the  class,  make  use  of  a  stop-watch.  Allowing  the 
point  value  of  the  word  for  correct  responses  and  half  that  value 
for  half  correct  responses,  grade  the  papers  according  to  the 
acceptable  responses  presented  in  Table  X.  Add  the  points  and 
record  this  sum  as  the  individual's  score. 

If  an  entire  class  is  to  be  compared  with  the  norms  herein 
contained,  compute  the  median  score  of  the  class  and  compare 
with  the  fifty  percentile  to  ascertain  whether  the  class  is  above 
or  below  standard.  The  ratio  between  the  percentile  score  of 
the  individual  or  of  the  class  and  the  fifty  percentile  of  the  group 
may  be  taken  as  index  of  brightness. 

Frequently  it  is  necessary  to  have  more  than  one  list  of  oppo- 
sites. To  meet  such  a  contingency,  the  list  has  been  divided  so 
that  both  parts  contain  an  equal  number  of  points  and  words. 
The  lists  are  given  below,  arranged  in  the  order  of  difficulty. 
The  assigned  point  value  is  at  the  left  of  the  stimulus. 

The  writer  feels  that  the  Opposites  Test  as  a  group  test  has 
sacrificed  to  time  an  important  element  in  that  it  fails  to  detect 
certain  aspects  of  the  subject's  mind,  aspects  upon  which  per- 
sonal efficiency  is  so  largely  dependent.  Most  significant  and 
enlightening    is    the   attitude   of    the    subject   toward    the    test. 

LIST  I 

I  Full  3  Despondent  5  Reveal  10  Analytical 

I  Dim  4  Doubtful  5  Join  11  Loquacious 

I  Blunt  4  Busy  7  Permanent  11  Sacred 

1  Pessimistic  4  Abstract  7  Obnoxious  12  Spurious 

2  Public  4  Foreign  7  Victorious  14  Facility 
2  Spend  4  Simple  8  Proficient  14  Pride 

2  Graceful  4  Extravagant         8  Rigid  17  Parsimony 

3  Ancient  4  Dangerous  9  Orthodoxy  22  Suave 
3  Barbarous            5  Defective 
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LIST  II 

I  Negative 

3  Vague 

5  Diligent 

9  Permit 

I  After 

3  Fertile 

6  Impoverish 

ID  Extrinsic 

I  Success 

4  Injurious 

7  Elation 

II  Dynamic 

I  Joy 

4  Advance 

7  Sinful 

12  Heterogeneous 

2  Profit 

4  Create 

7  Conservative 

13  Disastrous 

2  Always 

4  Aristocratic 

7  Obscure 

15  Result 

2  Strength 

4  Rare 

8  Repulsion 

15  Adroit 

3  Expand 

5  Slovenly 

9  Imaginary 

26  Exoteric 

3  Hinder 

5  Stingy 

Whether  these  attitudes,  which  Ruger^  has  entitled  subjective 
and  objective,  so  easily  perceived  by  the  experimenter,  carry  over 
into  other  lines  of  work,  vi^hile  highly  probable,  is  a  matter  that 
yet  remains  to  be  proved. 

The  poor  record  in  the  Opposites  Test  does  not  necessarily 
indicate  the  subjective  attitude  but  an  exceptionally  good  record 
does  more  probably  indicate  the  absence  of  it.  The  student  be- 
comes easily  confused  and  his  attention  is  divided  between  the 
opposite  to  be  given  and  fear  of  a  bad  showing.  The  score  goes 
low  on  account  of  frequent  interruptions  of  which  the  following 
are  typical:  'T  know  the  opposite  but  I  can't  think  of  it  right 
now,"  "I  can't  think  of  the  opposite  but  I  know  the  word  per- 
fectly," "This  is  a  vocabulary  test  and  my  vocabulary  is  limited." 

This  type  of  subject  seldom  admits  that  he  is  familiar  neither 
with  the  word  nor  the  opposite.  He  prefers  in  the  individual 
test  to  waste  the  allotted  ten  seconds  attempting  to  create  the 
impression  that  he  knows  the  opposite  but  this  time  he  can  not 
quite  get  it.  Frequently  he  forgets  the  present  stimulus  in  re- 
gretting the  one  just  missed. 

However  if  time  is  wasted  in  administering  the  test,  it  is 
more  than  counterbalanced  by  the  speed  with  which  he  makes 
his  exit.  He  does  not  tarry  to  ask  about  some  of  the  words  he 
has  failed  on,  nor  is  he  curious  about  the  records  of  others. 
With  a  hasty  explanation  of  his  own  inefficiency  he  hurries  from 
the  room,  leaving  the  experimenter  with  a  vague  sort  of  feeling 
that  the  latter  has  purposely  inflicted  a  mortal  injury. 

In  striking  contrast  is  the  subject  whose  interest  is  centered 
upon  the  test  itself.    He  works  rapidly  offering  neither  apologies 


iRuger,  H. 
XV. 


A.,  "The  Psychology  of  Efficiency,  Archiv.  of  Psychol.,  1910, 
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nor  explanations.  Unashamed  he  denies  knowledge  of  some  of 
the  more  difficult  words,  but  rarely  does  he  depart  without  this 
knowledge.  He  is  eager  to  know  what  others  are  able  to  do 
under  the  same  conditions.  Interest  in  self  is  overshadowed  by 
interest  in  the  problem  in  hand. 

Peterson,^  with  his  Rational  Learning  Test,  notes  these  same 
attitudes,  and  the  writer  knows  of  no  better  test  in  which  to  ob- 
serve them,  for  not  only  are  they  evident  during  the  administra- 
tion of  the  test,  but  the  traits  in  question  can  be  easily  detected  in 
the  individual  records. 

Determination  of  the  extent  to  which  the  opposites  test  reveals 
native  ability,  is  not  attempted  in  this  research.  It  is  hoped  that 
someone  will  carry  on  the  work  here  begun,  and  by  a  series  of 
correlations  empirically  determine  the  scope  and  limitation  of  the 
test.  The  writer  is  of  the  opinion,  that  if  used  with  other  stan- 
dardized tests,  the  Opposites  Test  will  be  found  of  value  in 
mental  diagnosis. 

2  Peterson,  Joseph,  "Experiments  in  Rational  Learning,"  Psychol.  Review, 
1918,  25,  462-463. 


XI. 

Summary  and  Conclusions 

I.  The  Hard  Opposites  Test  herein  presented  enables  the 
instructor  within  a  brief  period  to  examine  a  large  number  of 
students.  Even  the  most  inexperienced  will  find  little  difficulty  in 
evaluating  the  papers,  so  statistically  simple  is  the  test. 

II.  Initial  experimentation  began  with  all  words  previously 
employed  but  eventually  the  list  was  narrowed  so  as  to  include 
only  fifty-four  of  these  stimuli.  Similarly  the  fourteen  original 
words,  for  the  most  part  of  greater  difficulty,  were  selected  from 
a  long  list  which  was  subjected  to  the  same  process  of  investiga- 
tion. 

III.  Subjects  were  tested  both  orally  and  individually  so  the 
experimenter  might  study  each  stimulus  from  the  standpoint  of 
both  time  and  accuracy.  Records  were  kept  of  each  individual 
response  and  reaction  time.  On  the  basis  of  these  records  the 
relative  difficulty  of  the  words  was  computed.  The  calculations 
are  based  on  at  least  one  hundred  records  for  each  word,  varying 
up  to  one  hundred  and  eighty-seven  for  others. 

IV.  The  acceptable  responses  were  chosen  by  five  judges, 
from  all  those  responses  given  by  the  subjects  experimented  upon. 
Additional  responses  which  suggested  themselves  to  each  judge 
were  recorded  and  in  turn  passed  on  by  the  others.  Effort  was 
made  to  have  the  responses  include  every  possible  opposite,  not 
merely  the  most  common  ones.  These  judgments  were  averaged 
and  a  value  of  o,  5^  or  i  was  accorded  each  response. 

V.  The  relative  difficulty  of  the  words  was  determined  by 
multiplying  the  median  time  of  all  correct  and  half-correct  re- 
sponses by  the  percentage  of  failures,  allowing  one-half  a  point 
for  half-correct  responses.  These  values  were  then  converted 
into  standard  deviation  and  steps  of  1/5  standard  deviation  were 
marked  off.  Having  selected  for  the  easiest  word,  a  word  which 
for  college  students  approached  zero  difficulty,  we  assigned  to  it 
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a  value  of  one,  to  the  next  step  a  value  of  two.     The  hardest 
word  in  the  list  received  a  value  of  twenty-six. 

\'"I.  For  convenient  use  the  test  was  converted  into  a  group 
test.  The  stimuli  were  arranged  in  the  ascending  order  of  dif- 
ficulty and  printed  on  a  good  qualtiy  of  white  paper.  On  the 
back  of  the  sheet,  directions  make  clear  to  both  the  subject  and 
instructor  what  is  to  be  done.  Three  thousand  copies  of  the  test 
were  mailed  to  psychologists  in  normal  schools,  colleges,  and 
universities  throughout  the  country.  From  this  number  the 
records  of  1628  students  were  obtained,  on  the  basis  of  which 
norms  for  college  students  were  established.  These  standards 
are  presented  in  terms  of  percentiles  for  each  class,  including 
graduates,  and  for  each  sex. 

VII.  The  list  of  stimuli  has  been  divided  into  an  equal  num- 
ber of  words  and  points,  for  the  convenience  of  any  experi- 
menter who  may  desire  two  lists  rather  than  one.  As  it  stands, 
the  test  can  be  easily  incorporated  into  a  group  of  tests  and  com- 
parisons made,  provided  the  other  tests  are  presented  in  terms 
of  percentiles.  The  test,  because  of  its  large  number  of  possible 
points  and  its  difficulty,  offers  an  opportunity  of  testing  ade- 
quately a  less  highly  selected  group  than  that  represented  by  col- 
lege students. 

VIII.  The  results  of  this  investigation  tend  to  show  that  suc- 
cess in  the  test  is  dependent  upon  native  ability  rather  than  years 
of  schooling.  This  conclusion  seems  justifiable  in  view  of,  first, 
the  tremendous  amount  of  overlapping  among  the  several  classes, 
and,  second,  the  gradual  rise  of  the  lowest  scores  from  year  to 
year  while  the  highest  scores  remain  constant.  The  fact  that 
graduates  are  superior  to  seniors,  and  sophomores  to  freshmen 
is  to  a  considerable  extent  to  be  attributed  to  the  operation  of  the 
law  of  selection. 
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APPENDIX 


/ 


Name 

School 

Sex 

Underscore  the  class  of  which  you  are  a  member : 
Freshmen;     Sophomore;     Junior;     Senior;     Graduate. 

Directions  for  Giving  the  Test 

Before  distributing  the  test  blanks,  request  the  students  not  to  turn  over 
the  sheets  until  the  signal  is  given.  Ask  them  to  fill  in  the  blank  spaces  at 
the  top  of  the  page  and  then  make  clear  what  is  to  be  done  by  reading  aloud 
the  following  directions : 

On  the  other  side  of  this  page  are  a  number  of  words  beside  each  of 
which  you  are  to  write  as  quickly  as  possible  the  exact  opposite.  For  instance 
if  the  word  "Black"  occurs  you  should  write  White."  The  opposite  you 
write  must  belong  to  the  same  part  of  speech  as  the  word  in  the  list.  Phrases 
and  words  formed  by  prefixing  "Non"  are  counted  wrong.  Begin  at  the 
top  and  work  downward,  but  in  case  you  do  not  know  the  opposites,  pass 
on  to  the  next  word,  and  later,  if  you  have  time,  come  back  to  those  you 
have  omitted. 

You  will  be  given  just  six  minutes,  so  when  the  Instructor  says  "Ready" 
be  prepared  to  write,  and  when  he  says  "Go"  turn  over  the  sheet  and  begin. 
Continue  until  the  signal  is  given  to  stop. 

Full         ^..^^3JL;  Busy          tlU^  ^Obscure       tC4^>^ 

Negative  Abstract   c,-.^,-^i  Proficient     <M^tt* 

After         -^    -  Advance      •i-^.t'w^^  Rigid             f^s^-jt  "-^-V^ 

Dim      fc^<-«-  Foreign           "-••'*■  Repulsion     c^H^rM^tn^ 

Blunt     ^•^"^  Create       -^.^^  Imaginary    -rv-^ 

Success     W^*^  Simple      ^'.r'v^j.Xv*.  Permit           -rvf»k*A' 

Pessimistic    «^U'*.^i*'--^         Extravagant  Orthodoxy     Xt-^V/' A*'"} 

Joy            i^fo^  Aristocratic  Analytical      AY*"*^.'^*«»-f 

Public        l^T*-J*.  Rare            c»a*~w-»v»  Extrinsic        5-»*  Ut^.-*  vc. 

Profit        *-'^*  Dangerous  '^■^«^>--*"^  Sacred           '    ■.'■'• 

Spend        Seuvn.  Slovenly         ^  Dynamic 

Always       -.a-u-v*  Defective     ^'-^-^  Loquacious        SvXii-^-V 

Graceful     ■»    ••■>"  Stingy      ft-.i^..-*--»  Heterogeneous    ■?-'-■  ■^r'^^-'ne'^ 

Strength  Reveal      ^Cc^^  Spurious             -• 

Ancient     -v-r*^-^-  Diligent     -^-'^"V^'l.^  Disastrous  -t-u-^'r---- 

Expand      f«9-.-*>t>*Ar-  Join           -«v^<»»>«k'^'^  Facility      »l.v^»^-^ 

Barbarous     w«'&v^< '^  Impoverish    <,vx^<-v*'  Pride           -«~i«^ 

Hinder         '  -'^■^  Permanent    \,.&v-A "" *'"  Result       •^».«-mi- 

Despondent  Elation       •    -f^vnf^,-.  Adroit         «:£^a««»^ 

Vague        Jif-vt*.-**-  Sinful       vC^t^v»Avo  Parsimony      u^k^~*^^ 

Fertile         a^^ct-  , Obnoxious     ^>JU*«*^'^  ,'  Snave                .axi^o^v 

Doubtful     ^.^X^  Conservative  Jl'-'Wf*^'  Exoteric                 ■■^* 

Injurious      -'-'^■-'■■^'  Victorious     clil«i.».t»!Hi^ 
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